C++ gen (WIP)#626
Merged
Merged
Conversation
This currently doesn't work with ASan, but I have an open issue for this here: bdwgc/bdwgc#772
It leaks, leads to premature GC, takes too long to compile, and doesn't support runtime sorting functions. We'll need a new solution. For now, we're deep copying standard containers, which isn't going to last us very long.
This addresses a GC crash we were seeing, as documented here: bdwgc/bdwgc#800 (comment)
It's blocked on this Clang bug: llvm/llvm-project#146956
Shantanu can look into why it's failing.
This reverts commit e13558e.
It wouldn't allow me to do simple shit, so I wrote my own arg parser.
I didn't bring in the `-framework` usages, since those should not be hard-coded in jank. Instead, perhaps we need to provide an easier mechanism to forward flags to Clang. We do have `JANK_EXTRA_FLAGS`.
jeaye
force-pushed
the
cppgen+arc+bdwgc
branch
2 times, most recently
from
b07f22e to
bb7073d
Compare
jeaye
force-pushed
the
cppgen+arc+bdwgc
branch
from
bb7073d to
c95cb6d
Compare
We no longer use CLI11, so we just return 1 on failure.
This was referenced Jan 1, 2026
E-A-Griffin
pushed a commit
to E-A-Griffin/jank
that referenced
this pull request
Jan 14, 2026
* Explicitly set bdwgc debug flags * Move reusable_context to jtl::ref * Use gc allocator and remove thread_local usage * Upgrade bdwgc and use malloc redirection This currently doesn't work with ASan, but I have an open issue for this here: bdwgc/bdwgc#772 * Fix clang-tidy issues * Remove BppTree It leaks, leads to premature GC, takes too long to compile, and doesn't support runtime sorting functions. We'll need a new solution. For now, we're deep copying standard containers, which isn't going to last us very long. * Remove direct std::set usages * Hack in some cpp gen fixes * Just adding everything so I can have it on my mac * Address some tidy issues * Generate stack save/restore during loop IR gen This addresses a GC crash we were seeing, as documented here: bdwgc/bdwgc#800 (comment) * Fix more cppgen issues * Fix some more cppgen tests * More cppgen fixes * More cppgen fixes * More cppgen fixes * More cppgen fixes * More cppgen fixes * Don't wrap cpp_raw during eval * More cppgen fixes * Implement cppgen for letfn * Optimize core libs during phase 1 * Add cppgen for case * Skip static member test for now It's blocked on this Clang bug: llvm/llvm-project#146956 * Detect unsupported static member access * Add another static test * Fix tidy issues * Fix some string gc bits * Bypass one layer of typedefs manually * Ensure no implicit conversion for instantiation * Fix tidy issues * Remove remaining bpptree references * Add some missing meta cppgen * Intern ns in jank_load fn for cppgen * Remove bpptree entirely * Fix aot cppgen for vars * Fix var_ref cppgen * Clear up static const issue * Fix cppgen for throw * Fix inf/nan cppgen * Lift macro derefs too This was needed to support some `is` usages within the clojure-test-suite. * Test string_builder inf/nan * Improve profile output * Start optimizing cppgen for size 1. Dedupe lifted constants/vars across arities 2. Remove lifting of nil/bools 3. Remove baked in `using namespace` usages Much, much more to go. * Move lifted constants and vars to codegen We want to dedupe them by arity, which gives us just as much work in codegen as tracking them normally would, if not more. This also gives each codegen processor its own flexibility about which things to lift and at which scope to do it. * Use explicit empty containers in cppgen * Add empty string cppgen * More cppgen fixes * Fix var interning order for cppgen * Remove some dead code * Clean up some cppgen * Remove some dead cppgen code * Don't use namespaced strings for cppgen * Start changing module gen * Finish module-level cppgen optimizations * Further optimize cppgen * Properly gen all meta * Optimize cppgen size more * Try adding hard-coded inlining back * Prevent extra conversion from typed to untyped * Clean up thread id * Move immutable_string to system heap * Move string_builder to system heap * Switch native allocator and immer policy to system * Switch to reference counting * Fix more arc issues * Try fixing more arc issues * Fix cppgen for tail position defs * Remove nil global and thus constexpr * Fix cppgen .o loading with global ctors * Fix atom ref counting * Remove some more object* * Fix array map buffer overflow * Get arc working for phase 2 * Make object count atomic * Fix letfn cppgen and ratio test * Fix health check, C API, and AOT * Address some clang-tidy issues * Fix some Linux issues * Add more erasure to C++ gen * Hard-code nullptr qualification An LLM said that Clang doesn't consider this qualified, so it won't add it automatically. This works fine on libc++, but fails on libstdc++, due to implicit usings, most likely. * Add some cppgen retains, to be safe * Switch oref erasure to return an object_ref This allows us to more easily throw `object_ref`, rather than `object*`. * Remove arc It removed the bdwgc premature GC errors, but it led to Clang JIT compilation bugs instead. There's no way out! * Add bdwgc back * Upgrade bdwgc * Disable collection on macOS * Compile AOT modules using separate clang * Add CMake option for GC profiling * Add explicit libedit and libxml2 deps to flake.nix This MIGHT be needed, based on some issues I saw locally. We can revert if not. * Fix clang-tidy issues * Disable Linux ASan in CI again * Ignore more noisy clang-tidy bits * Disable a debug log * Document a bug * Allow for custom output targets; infer as needed This will set the stage for us to compile `clojure.core` to a C++ file, which we'll then compile right into jank for phase 2. This shouldn't be needed, but it gets us around some memory usage issues in CI with all of the C++ compilation we're doing. * Address clang-tidy issues * Build phase 2 from cpp source; add option for CI This will allow CI to cook `clojure.core` into jank directly from the C++ source, which will lower memory usage during compilation. It should also lead to better optimizations, so no complaining here. * Switch back to C symbols for jank_load fns It's technically UB to throw an exception across that boundary, but it's also a pain in the ass to deal with mangling. * Get aot e2e tests passing again I've put back core.o into libjank-stanalone.a, since AOT is broken otherwise. We'll see if CI still dies from memory issues. * Disable double? test for now Shantanu can look into why it's failing. * Add a todo * Fix some clang-tidy issues * Force phase 2 source builds for Nix * Revert "Force phase 2 source builds for Nix" This reverts commit e13558e. * Force phase 2 source building in CI * Skip linting on generated core cpp files * Skip Nix builds in CI for now * Add an additional Nix note * Oops, fix copy pasta * Limit jobs for release in CI * Work toward getting lein-jank going again * Remove CLI11 It wouldn't allow me to do simple shit, so I wrote my own arg parser. * Add CI swap * Carry over changes from jank-lang#631 I didn't bring in the `-framework` usages, since those should not be hard-coded in jank. Instead, perhaps we need to provide an easier mechanism to forward flags to Clang. We do have `JANK_EXTRA_FLAGS`. * Use a swap helper * Bump lein-jank version * Clean before building in CI * Fix cleaning to clear all old core libs * Fix some CLI flags * Fix a failing bash test We no longer use CLI11, so we just return 1 on failure. * Bump up CI swap space
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This resolves #598.