Zig's toolchain work is continually impressive. While I still don't plan to write software in it, given that I believe memory safety is table stakes, all of this stuff is very, very good. Before the incremental work, it was the toolchain and cross-compiler work. The toolchain stuff has continually been fantastic. I'm very curious to see what they come up with next!
> Semantic analysis is the most difficult part of the compiler to handle incrementally. Perhaps unsurprisingly then, this is where language design starts to matter a lot: while I am pretty confident that most modern languages could support incremental compilation similar to how we do, certain design decisions can make that much more difficult. Zig has had its design tweaked over the years (sometimes controversially) specifically so that it is easier to support fast incremental compilation.
This is something I wish that we had done with Rust. It is impossible to do all of the things at once, though, and we already had a tremendous amount of things to do. This is also part of the "when do you ship 1.0" tradeoff; for our goals with the language, 2015 was the right moment to launch, but if had a few more years to bake things, maybe we could have made compile times way faster. Software engineering is hard.
10 years ago, I commented on the Rust issue for "Incremental recompilation", where it was suggested that Rust could at least adopt Haskell GHC's model of incrementality, which is currently file-level:
I recommend anybody who's interested in incremental recompilation to read what GHC does, because the effort to achieve that is relatively low.
Of course there's always desire for more:
GHC currently needs to parse+typecheck+codegen a file before it can process other files that import it. Codegen is slow. Thus, there's currently demand split compilation into "stages", so that the next file can be typechecked after its imports have been just typechecked (not codegenned).
I would also enjoy if recompilation avoidance were to happen at the function level, not the file level.
Macro systems are a key language feature that can destroy incremental recompilation. In theory, Haskell is well set up for that, as its macro system (TemplateHaskell) is fully AST based and _theoretically_ could distinguish "fully pure" macros from side-effectful macros (such as splicing the current git commit in as a string literal). But the recompilation avoidance system does not currently exploit such differences.
Just to be clear about it, Rust today does do some amount of incremental compilation, and there is more work being done to continue to make it moreso. It's just very difficult to re-architect such a large and heavily used codebase. People are putting in heroic amounts of effort to improve things.
Rust's safety checks have basically nothing to do with its slow compile times. This is something that sounds intuitive but is just completely incorrect.
In particular, Rust made several good design decisions around this stuff that keeps those checks fast, like keeping checks local rather than being global.
That's interesting. Coming from C++ and Zig, the massive time "wasters" are metaprogramming features, i.e. Templates and comptime. Are Rust's macros the compile-time culprits?
not quite! if your project is mixed C/Zig, then editing Zig would work but editing C would not. the Zig compiler caches Zig AIR but no information from C. plus, the C compilation can only really be done with LLVM and writing a self-hosted backend would be PITA for C
It only works with our self-hosted code generation backends (the main one being for x86_64), which right now don't have any optimisation passes. It's planned that they will in future, but that's a long-term goal. Also, any optimisations which propagate information between functions (the most obvious and important one is inlining) are more-or-less incompatible with incremental compilation (I touch on this in the post IIRC), so once this does work it'll probably still be limited to a subset of optimisations.
TL;DR: only debug builds for now, could extend to release builds once we have our own optimisation passes one day, but some optimisations will still be inapplicable.
It disappoints me how unseriously the industry has taken compilation speed for so long. I'm glad to see Zig doing incredibly valuable, high-impact work here.
> Semantic analysis is the most difficult part of the compiler to handle incrementally. Perhaps unsurprisingly then, this is where language design starts to matter a lot: while I am pretty confident that most modern languages could support incremental compilation similar to how we do, certain design decisions can make that much more difficult. Zig has had its design tweaked over the years (sometimes controversially) specifically so that it is easier to support fast incremental compilation.
This is something I wish that we had done with Rust. It is impossible to do all of the things at once, though, and we already had a tremendous amount of things to do. This is also part of the "when do you ship 1.0" tradeoff; for our goals with the language, 2015 was the right moment to launch, but if had a few more years to bake things, maybe we could have made compile times way faster. Software engineering is hard.
https://github.com/rust-lang/rust/issues/2369#issuecomment-1...
This would already help a lot.
I recommend anybody who's interested in incremental recompilation to read what GHC does, because the effort to achieve that is relatively low.
Of course there's always desire for more:
GHC currently needs to parse+typecheck+codegen a file before it can process other files that import it. Codegen is slow. Thus, there's currently demand split compilation into "stages", so that the next file can be typechecked after its imports have been just typechecked (not codegenned).
I would also enjoy if recompilation avoidance were to happen at the function level, not the file level.
Macro systems are a key language feature that can destroy incremental recompilation. In theory, Haskell is well set up for that, as its macro system (TemplateHaskell) is fully AST based and _theoretically_ could distinguish "fully pure" macros from side-effectful macros (such as splicing the current git commit in as a string literal). But the recompilation avoidance system does not currently exploit such differences.
An example that's being funded right now: https://rust-lang.github.io/rust-project-goals/2026/expansio...
Zig doesn't have the same safety guarantees, it's on the dev to use safe coding patterns, so the tradeoff for safety is discipline or experience.
In particular, Rust made several good design decisions around this stuff that keeps those checks fast, like keeping checks local rather than being global.
TL;DR: only debug builds for now, could extend to release builds once we have our own optimisation passes one day, but some optimisations will still be inapplicable.
Getting incremental linker to work with llvm is kinda hard atm. Maybe the zig team has a plan dunno.