0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
zig/src/Module.zig

4971 lines
188 KiB

stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-28 19:38:19 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-28 19:38:19 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-03-28 19:38:19 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-05-07 18:52:11 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-05-07 18:52:11 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-05-07 16:05:44 -07:00
2021-05-07 18:52:11 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-05-07 18:52:11 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-01-02 19:03:14 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-28 19:38:19 -07:00
2021-03-28 19:38:19 -07:00
2021-05-04 13:58:08 -07:00
2021-03-28 19:38:19 -07:00
2021-03-28 19:38:19 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
2020-11-16 20:45:53 +02:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
2020-11-16 20:45:53 +02:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
2021-01-28 22:38:25 +02:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
2020-09-09 17:41:51 +03:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: support recursive inline/comptime functions zir.Inst no longer has an `analyzed_inst` field. This is previously how we mapped ZIR to their TZIR counterparts, however with the way inline and comptime function calls work, we can potentially have the same ZIR structure being analyzed by multiple different analyses, such as during a recursive inline function call. This would cause the `analyzed_inst` field to become clobbered. So instead, we use a table to map the instructions to their semantically analyzed counterparts. This will help with multi-threaded compilation as well. Scope.Block.Inlining is split into 2 different layers of "sharedness". The first layer is shared by the whole inline/comptime function call stack. It contains the callsite where something is being inlined and the branch count/quota. The second layer is different per function call but shared by all the blocks within the function being inlined. Add support for debug dumping br and brvoid TZIR instructions. Remove the "unreachable code" error. It was happening even for this case: ```zig if (comptime_condition) return; bar(); // error: unreachable code ``` We will need smarter logic for when it is legal to emit this compile error. Remove the ZIR test cases. These are redundant with other higher level Zig source tests we have, and maintaining support for ZIRModule as a first-class top level abstraction is getting in the way of clean compiler design for the main use case. We will have ZIR/TZIR based test cases someday to help with testing optimization passes and ZIR to TZIR analysis, but as is, these test cases are not accomplishing that, and they are getting in the way.
2021-01-02 22:42:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: support recursive inline/comptime functions zir.Inst no longer has an `analyzed_inst` field. This is previously how we mapped ZIR to their TZIR counterparts, however with the way inline and comptime function calls work, we can potentially have the same ZIR structure being analyzed by multiple different analyses, such as during a recursive inline function call. This would cause the `analyzed_inst` field to become clobbered. So instead, we use a table to map the instructions to their semantically analyzed counterparts. This will help with multi-threaded compilation as well. Scope.Block.Inlining is split into 2 different layers of "sharedness". The first layer is shared by the whole inline/comptime function call stack. It contains the callsite where something is being inlined and the branch count/quota. The second layer is different per function call but shared by all the blocks within the function being inlined. Add support for debug dumping br and brvoid TZIR instructions. Remove the "unreachable code" error. It was happening even for this case: ```zig if (comptime_condition) return; bar(); // error: unreachable code ``` We will need smarter logic for when it is legal to emit this compile error. Remove the ZIR test cases. These are redundant with other higher level Zig source tests we have, and maintaining support for ZIRModule as a first-class top level abstraction is getting in the way of clean compiler design for the main use case. We will have ZIR/TZIR based test cases someday to help with testing optimization passes and ZIR to TZIR analysis, but as is, these test cases are not accomplishing that, and they are getting in the way.
2021-01-02 22:42:07 -07:00
2021-06-06 21:08:31 +03:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: support recursive inline/comptime functions zir.Inst no longer has an `analyzed_inst` field. This is previously how we mapped ZIR to their TZIR counterparts, however with the way inline and comptime function calls work, we can potentially have the same ZIR structure being analyzed by multiple different analyses, such as during a recursive inline function call. This would cause the `analyzed_inst` field to become clobbered. So instead, we use a table to map the instructions to their semantically analyzed counterparts. This will help with multi-threaded compilation as well. Scope.Block.Inlining is split into 2 different layers of "sharedness". The first layer is shared by the whole inline/comptime function call stack. It contains the callsite where something is being inlined and the branch count/quota. The second layer is different per function call but shared by all the blocks within the function being inlined. Add support for debug dumping br and brvoid TZIR instructions. Remove the "unreachable code" error. It was happening even for this case: ```zig if (comptime_condition) return; bar(); // error: unreachable code ``` We will need smarter logic for when it is legal to emit this compile error. Remove the ZIR test cases. These are redundant with other higher level Zig source tests we have, and maintaining support for ZIRModule as a first-class top level abstraction is getting in the way of clean compiler design for the main use case. We will have ZIR/TZIR based test cases someday to help with testing optimization passes and ZIR to TZIR analysis, but as is, these test cases are not accomplishing that, and they are getting in the way.
2021-01-02 22:42:07 -07:00
2021-06-23 14:32:21 -04:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: support recursive inline/comptime functions zir.Inst no longer has an `analyzed_inst` field. This is previously how we mapped ZIR to their TZIR counterparts, however with the way inline and comptime function calls work, we can potentially have the same ZIR structure being analyzed by multiple different analyses, such as during a recursive inline function call. This would cause the `analyzed_inst` field to become clobbered. So instead, we use a table to map the instructions to their semantically analyzed counterparts. This will help with multi-threaded compilation as well. Scope.Block.Inlining is split into 2 different layers of "sharedness". The first layer is shared by the whole inline/comptime function call stack. It contains the callsite where something is being inlined and the branch count/quota. The second layer is different per function call but shared by all the blocks within the function being inlined. Add support for debug dumping br and brvoid TZIR instructions. Remove the "unreachable code" error. It was happening even for this case: ```zig if (comptime_condition) return; bar(); // error: unreachable code ``` We will need smarter logic for when it is legal to emit this compile error. Remove the ZIR test cases. These are redundant with other higher level Zig source tests we have, and maintaining support for ZIRModule as a first-class top level abstraction is getting in the way of clean compiler design for the main use case. We will have ZIR/TZIR based test cases someday to help with testing optimization passes and ZIR to TZIR analysis, but as is, these test cases are not accomplishing that, and they are getting in the way.
2021-01-02 22:42:07 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-06 21:08:31 +03:00
2021-06-23 14:32:21 -04:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-23 14:32:21 -04:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-04-20 17:03:18 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-20 17:09:06 -07:00
2021-03-21 19:23:12 -07:00
2021-03-21 19:23:12 -07:00
2021-03-21 19:23:12 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-04-08 20:52:02 -07:00
2021-04-08 20:52:02 -07:00
2021-04-08 20:52:02 -07:00
2021-04-08 20:52:02 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2020-11-21 21:12:33 -05:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 16:05:44 -07:00
stage2: implement simple enums A simple enum is an enum which has an automatic integer tag type, all tag values automatically assigned, and no top level declarations. Such enums are created directly in AstGen and shared by all the generic/comptime instantiations of the surrounding ZIR code. This commit implements, but does not yet add any test cases for, simple enums. A full enum is an enum for which any of the above conditions are not true. Full enums are created in Sema, and therefore will create a unique type per generic/comptime instantiation. This commit does not implement full enums. However the `enum_decl_nonexhaustive` ZIR instruction is added and the respective Type functions are filled out. This commit makes an improvement to ZIR code, removing the decls array and removing the decl_map from AstGen. Instead, decl_ref and decl_val ZIR instructions index into the `owner_decl.dependencies` ArrayHashMap. We already need this dependencies array for incremental compilation purposes, and so repurposing it to also use it for ZIR decl indexes makes for efficient memory usage. Similarly, this commit fixes up incorrect memory management by removing the `const` ZIR instruction. The two places it was used stored memory in the AstGen arena, which may get freed after Sema. Now it properly sets up a new anonymous Decl for error sets and uses a normal decl_val instruction. The other usage of `const` ZIR instruction was float literals. These are now changed to use `float` ZIR instruction when the value fits inside `zir.Inst.Data` and `float128` otherwise. AstGen + Sema: implement int_to_enum and enum_to_int. No tests yet; I expect to have to make some fixes before they will pass tests. Will do that in the branch before merging. AstGen: fix struct astgen incorrectly counting decls as fields. Type/Value: give up on trying to exhaustively list every tag all the time. This makes the file more manageable. Also found a bug with i128/u128 this way, since the name of the function was more obvious when looking at the tag values. Type: implement abiAlignment and abiSize for structs. This will need to get more sophisticated at some point, but for now it is progress. Value: add new `enum_field_index` tag. Value: add hash_u32, needed when using ArrayHashMap.
2021-04-06 17:43:56 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: improvements aimed at std lib integration * AstGen: emit decl lookup ZIR instructions rather than directly looking up decls in AstGen. This is necessary because we want to reuse the same immutable ZIR code for multiple generic instantiations (and comptime function calls). * AstGen: fix using members_len instead of fields_len for struct decls. * structs: the struct_decl ZIR instruction is now also a block. This is so that the type expressions, default field value expressions, and alignment expressions can be evaluated in a scope that contains the decls from the struct namespace itself. * Add "std" and "builtin" packages to the builtin package. * Don't try to build glibc, musl, or mingw-w64 when using `-ofmt=c`. * builtin.zig is generated without `usingnamespace`. * builtin.zig takes advantage of `std.zig.fmtId` for CPU features. * A first pass at implementing `usingnamespace`. It's problematic and should either be deleted, or polished, before merging this branch. * Sema: allow explicitly specifying the namespace in which to look up Decls. This is used by `struct_decl` in order to put the decls from the struct namespace itself in scope when evaluating the type expressions, default value expressions, and alignment expressions. * Module: fix `analyzeNamespace` assuming that it is the top-level root declaration node. * Sema: implement comptime and runtime cmp operator. * Sema: implement peer type resolution for enums and enum literals. * Pull in the changes from master branch: 262e09c482d98a78531c049a18b7f24146fe157f. * ZIR: complete out simple_ptr_type debug printing
2021-04-12 16:44:51 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
2021-05-07 18:52:11 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-06-19 21:10:22 -04:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
2021-05-07 18:52:11 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-05-07 18:52:11 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-22 23:46:51 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-01-19 00:34:44 +01:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-01-19 00:34:44 +01:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
2021-01-19 00:34:44 +01:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
2021-03-28 19:38:19 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-03-20 17:09:06 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: implement error notes and regress -femit-zir * Implement error notes - note: other symbol exported here - note: previous else prong is here - note: previous '_' prong is here * Add Compilation.CObject.ErrorMsg. This object properly converts to AllErrors.Message when the time comes. * Add Compilation.CObject.failure_retryable. Properly handles out-of-memory and other transient failures. * Introduce Module.SrcLoc which has not only a byte offset but also references the file which the byte offset applies to. * Scope.Block now contains both a pointer to the "owner" Decl and the "source" Decl. As an example, during inline function call, the "owner" will be the Decl of the caller and the "source" will be the Decl of the callee. * Module.ErrorMsg now sports a `file_scope` field so that notes can refer to source locations in a file other than the parent error message. * Some instances where a `*Scope` was stored, now store a `*Scope.Container`. * Some methods in the `Scope` namespace were moved to the more specific type, since there was only an implementation for one particular tag. - `removeDecl` moved to `Scope.Container` - `destroy` moved to `Scope.File` * Two kinds of Scope deleted: - zir_module - decl * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly changed to be a reference; this commit fixes it. Fewer ZIR instructions processed as a result. - declval_in_module is renamed to declval - previous declval ZIR instruction is deleted; it was only for .zir files. * Test harness: friendlier diagnostics when an unexpected set of errors is encountered. * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst on the last zir instruction in the block. Compile log implementation: * Write to a buffer rather than directly to stderr. * Only keep track of 1 callsite per Decl. * No longer mutate the ZIR Inst struct data. * "Compile log statement found" errors are only emitted when there are no other compile errors. -femit-zir and support for .zir source files is regressed. If we wanted to support this again, outputting .zir would need to be done as yet another backend rather than in the haphazard way it was previously implemented. For parsing .zir, it was implemented previously in a way that was not helpful for debugging. We need tighter integration with the test harness for it to be useful; so clearly a rewrite is needed. Given that a rewrite is needed, and it was getting in the way of progress and organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: entry point via std lib and proper updated file detection Instead of Module setting up the root_scope with the root source file, instead, Module relies on the package table graph being set up properly, and inside `update()`, it does the equivalent of `_ = @import("std");`. This, in term, imports start.zig, which has the logic to call main (or not). `Module` no longer has `root_scope` - the root source file is no longer special, it's just in the package table mapped to "root". I also went ahead and implemented proper detection of updated files. mtime, inode, size, and source hash are kept in `Scope.File`. During an update, iterate over `import_table` and stat each file to find out which ones are updated. The source hash is redundant with the source hash used by the struct decl that corresponds to the file, so it should be removed in a future commit before merging the branch. * AstGen: add "previously declared here" notes for variables shadowing decls. * Parse imports as structs. Module now calls `AstGen.structDeclInner`, which is called by `AstGen.containerDecl`. - `importFile` is a bit kludgy with how it handles the top level Decl that kinda gets merged into the struct decl at the end of the function. Be on the look out for bugs related to that as well as possibly cleaner ways to implement this. * Module: factor out lookupDeclName into lookupIdentifier and lookupNa * Rename `Scope.Container` to `Scope.Namespace`. * Delete some dead code. This branch won't work until `usingnamespace` is implemented because it relies on `@import("builtin").OutputMode` and `OutputMode` comes from a `usingnamespace`.
2021-04-09 23:17:50 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-04-12 21:31:48 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-04-12 21:31:48 -07:00
2021-06-19 21:10:22 -04:00
2021-04-12 21:31:48 -07:00
2021-06-19 21:10:22 -04:00
2021-04-12 21:31:48 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: inferred local variables This patch introduces the following new things: Types: - inferred_alloc - This is a special value that tracks a set of types that have been stored to an inferred allocation. It does not support most of the normal type queries. However it does respond to `isConstPtr`, `ptrSize`, `zigTypeTag`, etc. - The payload for this type simply points to the corresponding Value payload. Values: - inferred_alloc - This is a special value that tracks a set of types that have been stored to an inferred allocation. It does not support any of the normal value queries. ZIR instructions: - store_to_inferred_ptr, - Same as `store` but the type of the value being stored will be used to infer the pointer type. - resolve_inferred_alloc - Each `store_to_inferred_ptr` puts the type of the stored value into a set, and then `resolve_inferred_alloc` triggers peer type resolution on the set. The operand is a `alloc_inferred` or `alloc_inferred_mut` instruction, which is the allocation that needs to have its type inferred. Changes to the C backend: * Implements the bitcast instruction. If the source and dest types are both pointers, uses a cast, otherwise uses memcpy. * Tests are run with -Wno-declaration-after-statement. Someday we can conform to this but not today. In ZIR form it looks like this: ```zir fn_body main { // unanalyzed %0 = dbg_stmt() =>%1 = alloc_inferred() %2 = declval_in_module(Decl(add)) %3 = deref(%2) %4 = param_type(%3, 0) %5 = const(TypedValue{ .ty = comptime_int, .val = 1}) %6 = as(%4, %5) %7 = param_type(%3, 1) %8 = const(TypedValue{ .ty = comptime_int, .val = 2}) %9 = as(%7, %8) %10 = call(%3, [%6, %9], modifier=auto) =>%11 = store_to_inferred_ptr(%1, %10) =>%12 = resolve_inferred_alloc(%1) %13 = dbg_stmt() %14 = ret_type() %15 = const(TypedValue{ .ty = comptime_int, .val = 3}) %16 = sub(%10, %15) %17 = as(%14, %16) %18 = return(%17) } // fn_body main ``` I have not played around with very many test cases yet. Some interesting ones that I want to look at before merging: ```zig var x = blk: { var y = foo(); y.a = 1; break :blk y; }; ``` In the above test case, x and y are supposed to alias. ```zig var x = if (bar()) blk: { var y = foo(); y.a = 1; break :blk y; } else blk: { var z = baz(); z.b = 1; break :blk z; }; ``` In the above test case, x, y, and z are supposed to alias. I also haven't tested with `var` instead of `const` yet.
2020-12-31 01:54:02 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: inferred local variables This patch introduces the following new things: Types: - inferred_alloc - This is a special value that tracks a set of types that have been stored to an inferred allocation. It does not support most of the normal type queries. However it does respond to `isConstPtr`, `ptrSize`, `zigTypeTag`, etc. - The payload for this type simply points to the corresponding Value payload. Values: - inferred_alloc - This is a special value that tracks a set of types that have been stored to an inferred allocation. It does not support any of the normal value queries. ZIR instructions: - store_to_inferred_ptr, - Same as `store` but the type of the value being stored will be used to infer the pointer type. - resolve_inferred_alloc - Each `store_to_inferred_ptr` puts the type of the stored value into a set, and then `resolve_inferred_alloc` triggers peer type resolution on the set. The operand is a `alloc_inferred` or `alloc_inferred_mut` instruction, which is the allocation that needs to have its type inferred. Changes to the C backend: * Implements the bitcast instruction. If the source and dest types are both pointers, uses a cast, otherwise uses memcpy. * Tests are run with -Wno-declaration-after-statement. Someday we can conform to this but not today. In ZIR form it looks like this: ```zir fn_body main { // unanalyzed %0 = dbg_stmt() =>%1 = alloc_inferred() %2 = declval_in_module(Decl(add)) %3 = deref(%2) %4 = param_type(%3, 0) %5 = const(TypedValue{ .ty = comptime_int, .val = 1}) %6 = as(%4, %5) %7 = param_type(%3, 1) %8 = const(TypedValue{ .ty = comptime_int, .val = 2}) %9 = as(%7, %8) %10 = call(%3, [%6, %9], modifier=auto) =>%11 = store_to_inferred_ptr(%1, %10) =>%12 = resolve_inferred_alloc(%1) %13 = dbg_stmt() %14 = ret_type() %15 = const(TypedValue{ .ty = comptime_int, .val = 3}) %16 = sub(%10, %15) %17 = as(%14, %16) %18 = return(%17) } // fn_body main ``` I have not played around with very many test cases yet. Some interesting ones that I want to look at before merging: ```zig var x = blk: { var y = foo(); y.a = 1; break :blk y; }; ``` In the above test case, x and y are supposed to alias. ```zig var x = if (bar()) blk: { var y = foo(); y.a = 1; break :blk y; } else blk: { var z = baz(); z.b = 1; break :blk z; }; ``` In the above test case, x, y, and z are supposed to alias. I also haven't tested with `var` instead of `const` yet.
2020-12-31 01:54:02 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2021-06-19 21:10:22 -04:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-01-02 19:03:14 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-01-02 19:03:14 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
2021-01-02 19:03:14 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
stage2: caching system integration & Module/Compilation splitting * update to the new cache hash API * std.Target defaultVersionRange moves to std.Target.Os.Tag * std.Target.Os gains getVersionRange which returns a tagged union * start the process of splitting Module into Compilation and "zig module". - The parts of Module having to do with only compiling zig code are extracted into ZigModule.zig. - Next step is to rename Module to Compilation. - After that rename ZigModule back to Module. * implement proper cache hash usage when compiling C objects, and properly manage the file lock of the build artifacts. * make versions optional to match recent changes to master branch. * proper cache hash integration for compiling zig code * proper cache hash integration for linking even when not compiling zig code. * ELF LLD linking integrates with the caching system. A comment from the source code: Here we want to determine whether we can save time by not invoking LLD when the output is unchanged. None of the linker options or the object files that are being linked are in the hash that namespaces the directory we are outputting to. Therefore, we must hash those now, and the resulting digest will form the "id" of the linking job we are about to perform. After a successful link, we store the id in the metadata of a symlink named "id.txt" in the artifact directory. So, now, we check if this symlink exists, and if it matches our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD. * implement disable_c_depfile option * add tracy to a few more functions
2020-09-13 19:17:58 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00
stage2: type declarations ZIR encode AnonNameStrategy which can be either parent, func, or anon. Here's the enum reproduced in the commit message for convenience: ```zig pub const NameStrategy = enum(u2) { /// Use the same name as the parent declaration name. /// e.g. `const Foo = struct {...};`. parent, /// Use the name of the currently executing comptime function call, /// with the current parameters. e.g. `ArrayList(i32)`. func, /// Create an anonymous name for this declaration. /// Like this: "ParentDeclName_struct_69" anon, }; ``` With this information in the ZIR, a future commit can improve the names of structs, unions, enums, and opaques. In order to accomplish this, the following ZIR instruction forms were removed and replaced with Extended op codes: * struct_decl * struct_decl_packed * struct_decl_extern * union_decl * union_decl_packed * union_decl_extern * enum_decl * enum_decl_nonexhaustive By being extended opcodes, one more u32 is needed, however we more than make up for it by repurposing the 16 "small" bits to provide shorter encodings for when decls_len == 0, fields_len == 0, a source node is not provided, etc. There tends to be no downside, and in fact sometimes upsides, to using an extended op code when there is a need for flag bits, which is the case for all three of these. Likewise, the container layout can be encoded in these bits rather than into the opcode. The following 4 ZIR instructions were added, netting a total of 4 freed up ZIR enum tags for future use: * opaque_decl_anon * opaque_decl_func * error_set_decl_anon * error_set_decl_func This is so that opaques and error sets can have the same name hint as structs, enums, and unions. `std.builtin.ContainerLayout` gets an explicit integer tag type so that it can be used inside packed structs. This commit also makes `Module.Namespace` use a separate set for anonymous decls, thus allowing anonymous decls to share the same `Decl.name` as their owner `Decl` objects.
2021-05-10 21:34:43 -07:00