As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features).

> rust mostly-solved the memory management problem at compile time and without a GC

"Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware.

To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.

So sad dlang never gets the credit it deserves. None of these ideas in zig are novel.

For example, another language that immediately grabbed my attention (aspirationally; I haven't looked at it closely yet) is https://github.com/aardappel/goose/. That's not because it has arenas, but because it doesn't have anything else.

I don't understand your C++ example about it getting slower and needing to "move pointers."

Well, it isn't that and it doesn't require that.

> I don't understand your C++ example about it getting slower and needing to "move pointers."

Because low-level languages need to use machine pointers, their dynamic heap allocations have a high CPU overhead; it's that overhead that moving GCs are designed to reduce, but because they move pointers, using them requires an FFI between these pointers and machine pointers. This is why in low level languages we try to avoid dynamic allocations when we can, but that increases long-term maintenance costs.

That IS cool. I loved partial evaluation the first time I heard of it - Futamura projections, here we come! And if it can be used to obsolete language features I never was happy with (macros), so much better!

But until now, I never heard of it as a selling point for Zig. I assumed Zig was just yet another "C replacement but we don't want to deal with Rust's borrow checker". I will certainly have to take a closer look on Zig now. But this should be up and front in their self-promotion!

It is! "A fresh approach to metaprogramming based on compile-time code execution and lazy evaluation" is the second selling point after simplicity: https://ziglang.org

Obviously, they can't call it partial evaluation because not many people know what that is. Zig's approach was eye opening to me. I'm very familiar with how macros are used in Scheme, but comptime is intentionally weaker (unlike macros, it's referentially transparent, so strictly weaker) and I was surprised by just how far it can go. It's not everyday that you see a new kind of a partial evaluation construct, let alone a language that's almost entirely based on it (like Lisp only for comptime).

There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.

This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way.

There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need!

> which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower

What are you talking about? I've never encountered this and I've been using Rust for 10 years.

I think the idea is that a small program can organize its allocations and data structures to minimize number of calls to malloc, e.g. with preallocated workspace structs, or slab allocation, and similar approaches. But as a program gets bigger, there's a pressure to have looser coupling, to have subsystems with simple convenient APIs which leads to them doing on-demand malloc calls internally, rather than having consumers pre-allocate their needed workspace. Because that kind of workspace management results in more complex APIs and more burden on the consumer.

That said, I don't really believe it either, at least for the kind of codebase where it would matter (scientific computing, in-memory DB server, etc). A codebase that places an emphasis on minimizing heap operations in hot codepaths can do so by consistently using workspaces and allocation-avoiding APIs. I don't think it's so difficult really, but it does take a conscious design decision to do so. But writing something like a web browser in this way could be annoying due to most data having wildly variable sizes, and zig's arena concept would be very handy -- but rust has crates like bumpalo for that purpose.

My personal mantra: "Think in FORTRAN, code in Rust/Julia/C++". But I'm mostly working on HPC-style code where I don't have to do with wildly varying input or output sizes.

Except that's not composable - not only do you need specialised data structures, but all (transitively) allocating calls need to be specialised. That's the exact same issue we have in C++, and that's the issue Zig seeks to address. BTW, just the other day there was a post here about a language with another interesting approach, but I have yet to give it a close look: https://github.com/aardappel/goose/

> But I'm mostly working on HPC-style code where I don't have to do with wildly varying input or output sizes.

There you have it. The problems arise more quickly in concurrent rather than parallel code, and when there are lots of features added over the years that touch the hot paths.

> in-memory DB server

Actually, here there can be big problems (as it's also about concurrency rather than parallelism). Last week a colleague of mine looked at Moka and saw that it could only offer half the throughput as Java's Caffeine at the same latency and RAM footprint (almost; the Java program used 5% more RAM). When he looked into it, he saw that over 40% of the program's CPU was spent on the epoch-based reclamation.

Most crates for containers will be written such that the container types take an optional allocator type parameter that defaults to the global allocator. You can set it and it transparently uses the other allocator.

To improve the ergonomics, you'd define local aliases that use that allocator.

    type MyVec<T> = Vec<T, A = MyAlloc>;

   with_allocator(&arena, || {
      third_party_library::do_work()
   });

but custom allocators are rarely used in rust because they're simply not needed the vast majority of the time. if your language is not memory safe and you need to manage memory yourself, they're more important. but this isn't the case with rust.

c and zig folks are obsessed with arena allocators particularly because they can group lifetimes of individual objects, reducing the amount of malloc/free calls and thus the amount of use after free, double free, nullptr derefs, or leaks that can occur.

in rust this isn't a concern so custom allocators are only used for performance reasons.

but it turns out that in performance sensitive areas, you generally use custom data structures or those that already have their own allocation strategy baked in, like the generational_arena crate.

most of the time you are not calling thirdparty crates that allocate in performance-sensitive regions. either the crate is designed for this usecase and already uses a performant allocation strategy, or you're writing your own code here.

and in the rare case, you can trivially vendor the crate and pass your own allocator into it, or toggle the global allocator for callers.

but you also need to benchmark first before choosing an allocation strategy because it's not clear that a custom allocator will always guarantee better performance anyways.

and btw zig doesn't guarantee this anyways. you could pull in a dependency that instantiates their own allocator. at least in rust almost all crates use the global allocator as a default which lets you swap it out. if a zig dependency uses their own allocator the only recourse is to fork it.

rust doesn't have a performance problem, so any claims about it's custom allocator support leading to poor performance is unfounded. and thus so are claims about the superiority of zig's approach to allocators.

You say I outright lie for not mentioning the existence of something that doesn't exist??? I guess you're saying it's possible to create such a mechanism (or that some libraries do create ad-hoc ones), but that's not the point.

> there is nothing stopping you from using custom allocators with your own code or with calls to thirdparty dependencies

I didn't say there's anything in the language stopping C++ and Rust from having such a standard library and ecosystem of libraries. They just don't have that yet.

> if your language is not memory safe and you need to manage memory yourself, they're more important. but this isn't the case with rust. c and zig folks are obsessed with arena allocators particularly because they can group lifetimes of individual objects, reducing the amount of malloc/free calls and thus the amount of use after free, double free, nullptr derefs, or leaks that can occur. n rust this isn't a concern so custom allocators are only used for performance reasons.

This is simply untrue. I won't call it an outright lie, as it's probably just a lack of experience with low-level programming.

First, I'm trying to point out the problems we've had in C++, most of which only became apparent when evolving large codebases over time. People who have not had experience evolving large C++ or Rust codebases over years simply don't know about these problems and certainly can't claim they don't exist. Writing smaller programs in C++ or even large but young programs has always been a pleasure. The language is expressive and productive. Some of the biggest issues only arise years later, when the program gets either expensive to maintain or slow.

Second, experienced C and C++ folks cannot be "obsessed" with arenas for the reasons you mentioned because until maybe 20 or even 15 years ago memory safety wasn't a widespread obsession. It was a correctness issue like all others, and its outsized role as the cause of security vulnerabilities wasn't widely known until more recently.

Lastly, you don't pick Rust for safety. Most software in the world today is already written in languages that are at least as memory-safe safe as Rust, sometimes more so. These days, you pick C, or C++, or Rust, or Zig when you want to do something that's largely low-level. Things that are low-level often also need to be reasonably fast, and large low-level codebases that evolve over years tend to suffer serious performance issues because of memory management (because, being low-level, they can't move pointers and so can't use things like a moving GC to reduce the overheads of their malloc/free runtimes; this is why companies with actual experience with long-maintained large low-level codebases make huge runtimes like TCMalloc to help them to a degree, which you also may not have needed yet), and arenas are the primary way to get memory performance similar to what you see with modern moving GCs (and even somewhat better).

Now, you could say that C++ only started moving in that direction with pmr in C++ 17, and that's true. But the need was recognised as early as 2005, traditionally C++ codebases didn't rely on many libraries so interoperability has typically not been a large concern, and the number of large C++ programs that would benefit from such a thing declined over the years because of the low-level maintenance issues I mentioned and the growing availability of fast high-level languages.

My distaste for Rust isn't because I like C++ so much. Even though it's been one of my primary programming languages for the past 25 years, I "hate" it for the very same reasons. Most Rust superfans are people who have not had enough experience with it and they don't know about the problems. Not all, of course, and even C++ has superfans, which is why I said that among the people who are experienced in low-level programming, there are people who like the C++/Rust approach (of trying to make low-level code appear high-level) and people who don't.

We could use rust. But it wouldn’t give us anything.

I'm curious why you think that statement is doing heavy lifting. It's much easier to write and verify that a few lines of code are correct than it is to write and verify that an entire program is correct. But that's the norm in C and Zig, and historically people haven't been very good at it. That's why we try to do it as little as possible.

The reason it's not "the norm" is that (especially with spatial safety taken care of), not every line is equally dangerous at all. Still, there's no doubt that more guarantees help, but that is only when all other things are equal. If you pick a low-level language for mostly low-level things, so Rust doesn't offer safety for the trickiest code, and furthermore it makes certain things harder to see because the language is more complicated, then things become much less clear. Obviously, when the vast majority of the trickiest, most important code doesn't need to be low-level, Rust would probably be safer on the whole, but in such situations I see no reason to choose either Rust or Zig. You need to choose a low-level language if the core of what you're doing needs to be low-level.

Is there a word missing before "memory"? Seems odd to specifically call out spatial memory safety when memory safety subsumes it.

You must reason about the invariants in unsafe code on a global level. In particular, you could have unsafe code in crate A, whose data are then used by crate B. It could be fine. But then crate B changes its implementation which now violates the invariant expectations of crate A.

Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A.

Also for what it's worth I get the impression you and treyd might be talking about slightly different things when talking about whether unsafe code composes. I believe treyd is referring to the RustBelt series of papers [0, 1], for which the statement "unsafe code composes" means (at a high level) that adding a module with a memory-safe API to a memory-safe system will result in a memory-safe system as long as the implementation upholds the safe semantics. Yes, the last bit can be a rather significant caveat, as you said.

What you're talking about seems more along the lines of needing to look beyond the boundaries of unsafe blocks to prove that the unsafe block upholds its invariants, which is also true. I think you only need to check within whatever safe encapsulation boundary is relevant, though, rather than globally.

[0]: https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf

[1]: https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf

This is a specifically crafted bad idea, but you could have module A use unsafe to craft a Vec<u8> that is safe to use to read or write, but not to grow or shrink. You declare an invariant that the receiver shalt not grow or shrink the Vec.

If B only reads and write, you're good. But if a future B breaks the invariant, bad things happen. As I said, specifically a bad idea; there's a much better type to use if the thing can't grow or shrink...

No real world example, because I don't think we've run into memory safety issues with unsafe in the Rust code base I work in... but we only use unsafe where it's required (syscalls and other FFI).

To address the problem that once integrity can be violated anywhere, only global analysis can prove that nothing bad happens, we've done two things:

1. We require the application to explicitly permit any integrity violation by a module; i.e. a library can't allow itself to violate integrity. This is a principle we call "Integrity by Default" (https://openjdk.org/jeps/8305968).

2. We try to minimise the need for potential integrity violations (this is very different from Rust, which requires unsafe even for things like benign write/write races, which are fairly common, and various basic data structures). Over the years we've offered safe replacements for things that used to require Unsafe. In other words, clearly demarcating unsafe code isn't enough if it's needed at all in many situations.

It isn't perfect, of course, as some libraries do require unsafe operations for direct interaction with native code or with memory, but their number has been greatly reduced, and they cannot do this without the application's explicit approval. Interestingly, this has annoyed library authors who want to do unsafe things but don't want to application authors to be alarmed because "we know what we're doing," and it's also annoyed some application authors who want to use such libraries and are forced to explicitly add permissions. But I think that the community, as a whole, has eventually accepted this because the harm done to those who don't care is small (they just need to add the permissions), to those who do care it helps a lot, and because fewer and fewer libraries require "integrity-busting" permissions, many applications need to do absolutely nothing and get important guarantees for free.

I know this paper [0] is quite old at this point, but the mention of benign data races reminded me of it. Would you happen to know how applicable it is to modern memory models?

[0]: https://www.usenix.org/legacy/event/hotpar11/tech/final_file...

However, in C and C++ (and Rust) benign non-atomic write/write races are UB (indeed, LLVM also treats them as potential causes of UB). In C# and in Java they are safe (although Java currently only has non-atomic writes on 32-bit machines, but soon they'll be more common when value types are enhanced). LLVM even has a specific construct to support the Java-style memory model (https://llvm.org/docs/Atomics.html#unordered), and Zig lets you use it (https://ziglang.org/documentation/master/#atomicStore).

It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.

I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.

Suppose you're writing a program that's mostly high-level, say some kind of concurrent server, and it's large-ish, say around 1MLOC (most C++ programs I've worked on were significantly larger). Because the language is also a low-level language, it has low-level constraints, so:

1. It needs to use an AOT compiler, and consequently to get good performance you need to use less general mechanisms, such as direct (as opposed to dynamic) dispatch and even manual monorphisation (with generics/templates). These are viral, so they have to be carefully chosen (you can't monomorphise everything or you'll get machine code explosion). Five years later you need to make a big change that requires more generality, and then you either have to reconsider all of your manual optimisations, which is expensive, or go for more general constructs (dynamic dispatch) and the program gets slower.

2. It needs to use machine pointers (i.e. you can't enjoy a moving GC), and so you try to use the stack as much as possible (which you can't really do for anything dynamic), or suffer the high cost of malloc/free on individual objects. As the program evolves, you need to make things more general, and objects that could live on the stack now need to go on the heap, and objects that lived on the heap now may need to be shared among threads, in which case you often add the additional cost of refcounting GC. Of course, you want to use arenas in many cases, but they're very, very hard to use in C++ and Rust.

You'd be better off - performance-wise and maintenance-wise - with a good optimising JIT and a moving GC. This was exactly a problem with many C++ programs that didn't really need a lot of direct hardware interaction - everything worked great for a few years, and then the evolution and maintenance costs became really high (or the programs became slow).

Now suppose you're writing something low-level, i.e. you really need to interact with the hardware and/or OS directly a lot, and want to control everything - where everything is in memory, exactly when it's initialised, exactly when it's freed, exactly which operations are executed and when. But now you have a language that's also high-level, so it has a lot of implicitness that hides from you the things you want to see (and in Rust's case, you lose the safety). Best case scenario, you rely on disciplne and avoid implicit features, but then you also need to avoid much of the standard library.

Anyway, combining high and low level in the same language was C++'s dream: one language for everything. Of course, for a while we didn't know about the maintenance problems, as those appear only years down the line, but more importantly, there weren't really high-performance high-level languages back then. These days, with lessons learnt and with more options, I prefer a language that focuses on being high-level for high-level stuff, and a language that focuses on low-level for low-level stuff. If you really need both kinds, use two languages.

You haven't been doing any Rust though. You seem to think you can extrapolate your C++ experience to Rust. That's preposterous. The actual Rust programmers can't recognize this theoretical problem in their Rust programs.

Most actual Rust programmers haven't maintained a large Rust program for a long time. Now, don't get me wrong - there are many C++ programmers who are fine with it, but many who aren't. What I find annoying is people without much experience in Rust assuming that everyone or almost everyone should like it, even though that's never been true for any language. I'm not saying Rust is bad by any means; in fact, I think it's better than C++ in a few ways. I'm explaining why I don't like it.

And if you look at other comments in this thread, many engineers have this mentality of "just use a crate, it's probably optimised already". They might not have performance problems immediately or obviously but it's more like ten thousand papercuts - a few allocations here and there, a few extra copies here and there and you've got a way slower program than it should have been.

Why? Won't an AI that can correctly write any program (and make any change) also be smart enough to know what exactly we want better than we can explain, at least ahead-of-time?

If AGI means "human level", why is there any part of the process that humans will be needed for, especially some engineering aspect?

> With proofs, we can verify that the AI implemented our prompts correctly.

Certainly such an AI would be able to just write machine code directly and verify it through whatever means, including formal proofs, as needed. Why does it need a compiler?

I think that an AI that's smart enough to write almost any program and prove almost any property, will also be smart enough to not need to communicate with us formally and rather answer every question we have (and proofs are not always necessary, as they're not always necessary today), and probably also smart enough to figure out what we want built. It's probably capable enough to replace the software's users, too. I don't understand why it's likely that we'll have AI that's so capable to write all software correctly, yet not capable enough to do things that are probably easier.

If the AI can write the program bytecode through AI magic, why can’t it verify that it works through AI magic?

The AI needs a compiler for the program for same reason it needs a proof language to verify it.

Why would an advanced intelligence be able to think about large programs without similar tools?

Moreover, why do you believe the AI will be able to write the program without special languages, but that it would still need a special language to verify the program?

I've done my share of formal verification (see my old blog in my profile) and of programming in Assembly, and obviously the former is far harder than the latter. People have only managed to formally prove the correctness of programs (end-to-end) under ~10 KLOC. If for some reason an AI capable of writing and proving arbitrary programs thinks it will be aided by a compiler, surely it will be able to create its own ad-hoc one for the problem at hand.

Also predicate logic can be expressed using machine code, so why does the AI need a whole other notation to express predicate logic? It should just be able to use whatever machine code it used to write the program.

If the answer is that the verifier would benefit from extra semantic information in the logic language, then why doesn’t that same representational argument apply to programs? Programs also have semantic structures that aren’t naturally expressed by the ISA. E.g. types, effects, state machines, matrix operations, synchronization, ownership, capabilities, protocols, etc.

Moreover, why can’t this hypothetical AI just write correct code that doesn’t need to be proved through predicate logic?

> If for some reason an AI capable of writing and proving arbitrary programs thinks it will be aided by a compiler, surely it will be able to create its own ad-hoc one for the problem at hand.

I think this basically concedes my point: you’re saying the ai will create ad hoc programming languages to support its thinking, which I think is much closer to what will happen. I think it would choose to do this in nearly all cases. It’s one thing to be able to prove arbitrary programs, quite another to do so within resource constraints, like finishing the proof before the heat death of the universe.

Take for instance this expression:

A * B

Where A and B are matrices and * is matrix multiply.

The most efficient machine code for this will unroll all of the loops and multiplications and additions, pipeline and fuse them, and schedule them across parallel registers for SIMD instruction.

The machine code instructions to express just this one instance would be hundreds to thousands of bytes depending on the size of the matrix.

What’s easier: verifying the high-level matrix multiply to machine code lowering is correct (per machine) and therefore every application of it is correct; or that every single bespoke matrix multiply actually correctly implements matrix multiply for every machine?

At some level you’d spend more time proving every instance of bespoke unrolled matrix multiply machine code every time you generate it than just generating a permanent deterministic proven correct compiler and using that instead. I think a sufficiently smart AI would figure this out and absolutely take that shortcut, for the same reasons humans did.

The prover itself benefits from establishing higher level concepts as well, so there’s a real disconnect between the predicate logic and the machine code. The verifier would thrive on semantic information like: the matrix dimensions are fixed, the dimensions are compatible, the types are consistent, the values are immutable… all of that is intentionally missing by the time you lower to machine code (usually because it’s usefulness in proving the program have been exhausted). It would be better to hand these as statements of fact (determined by a type checker) to the verifier rather than asking the verifier to first prove them from byte code and then do the actual correctness proof.

Removing all that semantic information leaves the prover having to reinvent it on every proof. So I guess you could include that higher level semantic information in the machine code but now you’re back to designing a programming language.

Why would any other specific language be? Machine code is the most general and, as I said, if the AI wants to use a compiler to, say, help generate code for other platforms, it can create one.

> Also predicate logic can be expressed using machine code

It can't. Predicate logic has certain deduction rules, and these rules are different from those of machine code (e.g. machine code doesn't have quantifiers or their introduction and elimination rules). Of course, the AI could invent an ad hoc logic language and write a prover for it, but in this thought exercise, I think the point is that people would want to verify the proven properties (and possibly the implementation of the proof checker).

> What’s easier: verifying the high-level matrix multiply to machine code lowering is correct (per machine) and therefore every application of it is correct; or that every single bespoke matrix multiply actually correctly implements matrix multiply for every machine?

This isn't too relevant. If you need to build a skyscraper for tens of millions of dollars, you wouldn't spend much time thinking about how to lower the cost of the handle on the front entrance by $10. Even if every matrix multiplication is a little different, the proofs can be copied and changed slightly. The difference in cost of doing that or not is negligible compared to other aspects of the verification. Again, we write programs that are three orders of magnitude larger than the largest program we've ever managed to prove correct, and the difficulty does not scale linearly. So we're talking about an AI that's 10,000 or 100,000 times more capable than humans at writing software. Surely, matters as duplicating code and proofs are negligible for it.

But even if it does matter, machine code isn't a problem because you don't have to duplicate manually. You can write the machine code that specialises the different applications and prove the correctness of that. This is like writing a small portion of a compiler.

My general point is that proving arbitrary (practical) correctness properties of arbitrary (practical) programs is such a monumental achievement that anything more mundane, like a compiler, is trivial in comparison. It's like saying that someone can boil an ocean or move a mountain, but we must order them a taxi to get there.

"We are a Java house" is not a Greenfield thing. It's picking what you already use internally.

Everything that was done on the backend side was done in Java, although there were some exceptions for .NET deployments.

So we ended up with mixed skills teams where depending on the ticket, you would be coding Java or C#.

medtech - .NET and TypeScript.

Second, I wonder where, roughly, would you put a transition from small programs where low-level langs are fine, to programs large enough to heavily benefit from JVM tradeoffs? And how this transition is affected by a stuff like Graal Native?

As to Rust - we all, hopefully, agree that it’s great language, but not for some startup making websites or Mongo based, boring backends. It’s great for the stable, system level products.

I'm not following. Saying Go lacks in X shows how poor other software is? Can you connect the dots?

We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly. I actually love both languages.

I'm well aware that Go's GC has improved, but the moving algorithm was designed not just to be fast for a GC, but to be faster than no GC. So Go's new GC is good - for a mark and sweep collector. But it can't compete with a moving collector (the only thing that can is arenas, which are user-friendly only in Zig).

> We are using JDK25 and are considering rewriting parts of our product to Go because of lower memory pressure and faster startup time, i.e., cloud friendly.

Java probably will never have perfect warmup, but it's getting very good - https://openjdk.org/jeps/544 - probably in JDK 28.

As for memory, I think Java's memory strategy is generally misunderstood and I've given a talk about it: https://youtu.be/xr73mR7ii9M The footprint overhead exists to compensate for CPU utilisation when the CPU utilisation is more disruptive than memory usage. The problem is that many Java developers - and I'm not blaming them - don't understand this tradeoff and how to configure the JVM for optimal resource usage, but the great news is that a solution is coming soon, too - https://openjdk.org/jeps/8377305 - also possibly in JDK 28.

So it's very likely that both of these issues will be resolved six months from today, and you'd still get to enjoy better performance and telemetry than all alternatives.

The amount of memory a Java program uses is whatever the setting is, not how much it "needs", because the need depends on the preference of the CPU/RAM tradeoff. But again, not many understand that, so we're making that automatic.

This is not true. The whole point of the algorithm - the reason it was designed - is that the amount of moving is well below what's required in a non-moving collector. The downside is that the algorithm is more complicated and requires an FFI layer for FFI, but even though non-moving collectors are far simpler to implement, every language/runtime that can use moving collectors uses them (and all of those can also use non-moving collectors, too, as Java did earlier on; concurrent mark-and-sweep collectors like Go's or Java's old CMS are easier to make). Whatever you say about the complexity of moving collectors or their impact to latency before the recent invention of pauseless moving collectors, they are widely recognised fact that as the most efficient general purpose memory management solution (but also the most elaborate).

You could argue about certain workloads, but it is ridiculous to claim that the world's top memory management researchers worked for years to come up with an algorithm to be more efficient than mark-and-sweep collectors and malloc/free failed to notice that it has to move objects around a lot (the whole point of the algorithm is that it does not), and then every language that can use the algorithm chooses to use it because they also failed to notice that the algorithm that is so much more costly to implement is so obviously worse.

BTW, Go's reason for using a simpler, older style mark-and-sweep collector isn't that it's better (Google's larger V8 team opted for a moving collector), but that Go can get away with a simpler, less efficient GC because the allocation rate is lower (and we can argue over that, but at least that would be an argument over something that could actually be controversial).

Anyway, if you're interested to know how moving collectors really work, and how they were created to be more efficient than any non-moving general memory management strategy, I go through the basics in a recent talk I gave: https://youtu.be/xr73mR7ii9M

Lower memory pressure is certainly a difficult thing to beat Go at, Java (OpenJDK) is probably never gonna get there. You get a lot of other stuff, like better peak performance, instead.

Btw, have you tried Leyden/AOT for better startup times? Curious about your experiences with that.

Nope, not yet. It's a good question given that up to now we used to deliver our product only on-premises and Windows Server-only, but this year we are now finally going with the Cloud, which means Docker containers and Linux.

If I remember correctly Leyden required some sort of warm-up and training data collection before being able to effectively execute AOT, right? I need to freshen up my info on that.

I did try GraalVM-compiled Java executables a couple of years ago and they were not bad, but the binaries were quite big (not a showstopper though) and the class-loading issues were kind of a PITA.

It's usually the build systems that add quite some overhead.

Which specific optimizations are you referring to?

In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away.

JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null).

A JIT with speculative optimisation and a moving GC.

There are two constraints in low-level languages that trump any of their performance goals, one technical and one a matter of preference.

The technical limitation is that they must use stable pointers (because they need to be low-level and so having an FFI layer that separates "hardware pointers" from "language pointers", as we have in Java defeats their main purpose). This means that you need to translate data storage or code storage to hardware addresses, and that interferes with both moving collection and with JIT compilation.

The other constraint is that low-level languages value worst-case performance over the average-case and even amortised performance. These languages prefer an operation (e.g. dynamic dispatch) to be slow as long as it's never too slow. With a JIT (and I describe more later), virtual dispatch can be super-fast almost all the time, but occassionally, you'll hit a trap because the speculation was wrong, and then you need to deoptimise and recompile.

> In my experience, this is largely a myth; compared to Rust, you actually get even faster code right away.

We wouldn't be doing it in the first place if it was a myth. In a low-level language, you can get very fast code if you do some manual optimisations, but they don't easily scale as the program grows and evolves, because they're viral. The two most basic examples are dynamic dispatch (which is the most general mechanism, which scales the best in terms of program evolution) and shared heap objects (again, the most general mechanism). These become more common and less easily avoided over time, and they're slow in low-level languages because of the constraints I mentioned.

That low-level languages make it harder and harder to preserve good performance over time as they evolve and grow is a problem familiar to those who've worked for years on large software written in a low level language (as I have). The JVM was designed, among other things, to solve this performance problem in large programs.

> JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null).

A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do. E.g. by default, Java inlines and specialises virtual calls 15 levels deep. An AOT compiler can't do that or its code will explode. We get around it with selective use of templates in C++ (or comptime in Zig), but it has to be selective, and it's viral.

Do you mind a reasoned discussion?

> A JIT with speculative optimisation and a moving GC.

Idiomatic Rust, through its concepts of ownership and borrowing, encourages a pattern where you receive data as an argument or create it directly, perform operations on it, and then discard it via RAII. This bears some resemblance to functional programming. This approach does not apply to buffers of unknown size, which still require heap allocation; unfortunately, Rust lacks automatic buffer reuse. However, such optimization is theoretically possible. The stack is definitely faster than anything else.

> This means that you need to translate data storage or code storage to hardware addresses, and that interferes with both moving collection and with JIT compilation.

You don't need GC if you allocate data on stack. You also do not need to dereference the pointer.

> dynamic dispatch

You mentioned templates. In Rust, traits that are monomorphized - much like templates-are the standard approach; using vtables or `dyn trait` is a relatively rare use case. This stems from the fact that all code is known at compile time and there is no dynamic loading, allowing the compiler to eliminate polymorphism from the code entirely.

> and shared heap objects

This might be considered convenient, but in my view, it also leads to code that is harder to maintain when objects can be modified from multiple places. However, I think that is outside the scope of the current discussion.

> We get around it with selective use of templates in C++ (or comptime in Zig), but it has to be selective, and it's viral.

Yes, monomorphization is the default solution in Rust. It is not always viral either, because when using it, you often define specific types, and they do not spread beyond that scope.

I suppose you could say that the programming style I am talking about is complex, inconvenient, unmaintainable, and so on. What I mean is, assuming this programming style is sufficiently convenient—and perhaps even has its own advantages - then none of the optimizations you listed offer an edge, and the Rust code will definitely be faster.

I have seen it mentioned everywhere, but is this actually true?

I mean, of course it is faster than random cold memory, but is it actually faster than a hot, in-cache part of the heap? It is not special in any other way, AFAIK.

And for what it's worth, what pron mentioned, Java uses a pretty similar structure for initial allocation, a thread local buffer where you just pointer bump. Another thread can then in the background copy still alive objects from this "arena" and then reset the whole thing.

Yes, it just adding or subtraction int to stack pointer register. I’m not certain, but the only thing that might be faster is accessing data at a fixed address - that is, global variables.

Stack is fast because it is frequently "touched" staying in cache. If you were to continuously read write a small segment of the heap, I don't think it would fair any worse than "the stack". This was my point

What you're describing isn't a stack, but an automatic arena, and this optimisation is easier to do in Java. It's easier to do in Java because it requires setting a "current arena" or inlining, both of which Java can do more easily, and then either the arena will be heap allocated (which will be slower in Rust) or associated with the thread, which is not something low-level languages tend to do.

> You don't need GC if you allocate data on stack. You also do not need to dereference the pointer.

Moving collectors don't need to dereference anything (they don't know and don't want to know when an object is "dead"), and stack allocation works in both languages, only, as you pointed out, is not quite general (not every data structure with a known lifetime can be allocated on the stack).

> You mentioned templates. In Rust, traits that are monomorphized - much like templates-are the standard approach; using vtables or `dyn trait` is a relatively rare use case. This stems from the fact that all code is known at compile time and there is no dynamic loading, allowing the compiler to eliminate polymorphism from the code entirely.

Sure, except Java does this automatically, and it can do it more aggressively. Dynamic dispatch is rare in low-level languages because it's expensive in those languages. But it's not easy to avoid as programs get larger. That is exactly one of the problems in large programs that the JVM set out to solve.

> This might be considered convenient, but in my view, it also leads to code that is harder to maintain when objects can be modified from multiple places. However, I think that is outside the scope of the current discussion.

I agree that whether it has downsides is outside the scope of this discussion, but the point is that as programs evolve and grow, the abstractions tend to be more general, and low-level languages suffer from "abstraction cost", where the more general abstraction (which becomes more common over time) is more expensive. Again, this is exactly why large C++ programs suffered from performance issues and what the JVM tried to address.

> Yes, monomorphization is the default solution in Rust.

... and in C++. But it is viral, and Java monomorphises without suffering from "zero overhead abstractions".

The ability to move pointers, both to data and to code, opens up the possibility of using JITs and moving GCs, which are very powerful optimisations. A JIT does impose two further tradeoffs (aside from the need for an FFI layer), though, which are warmup and the possibility of deoptimisation. We can now cache the generated machine code from one execution to another (https://openjdk.org/jeps/544), but the possibility of deoptimisation remains (in fact, it's what enables the aggressive speculative optimisations), which means you gain average (or even amortised) performance at the cost of the worst case.

Anyway, the JVM was designed as a solution for the performance issues low-level languages suffer from as programs grow and/or evolve. It comes with tradeoffs, but those most affect small or short-lived programs.

The thing to remember is that low-level languages are not optimised for performance but for low-level control (i.e. pointers are direct addresses etc.). Such control can translate to good performance when programs are small (see next) but it becomes a practical hindrance to performance when they're large.

> I suppose you could say that the programming style I am talking about is complex, inconvenient, unmaintainable, and so on. What I mean is, assuming this programming style is sufficiently convenient—and perhaps even has its own advantages

That advantage is a performance advantage. The question isn't "does there exist (in the mathematical sense) some program that is fast?" but "how fast is the program we can write within the budget we have?" When programs are small, manual optimisation is practical; when they grow large - not so much. And that's excluding the matter of a moving collector, which is just hard to compete with on speed regardless of program size, unless you use areans, but they're not at all easy to use in most low-level languages except Zig.

> and the Rust code will definitely be faster.

This is true only in the abstract mathematical sense. The reason we don't write programs that we want to be fast in Assembly (which is faster than anything in the same sense: for any program in any language, there exists and Assembly program that's at least as fast) is not because other languages are fast enough, but because in practice the programs we can actually write in the budget we have will be faster than the Assembly programs we could write. Of course, that could change when AI is able to generate perfect low-level code, but when that happens, it might as well generate machine code directly.

At least you aren't claiming that the JVM is ~1.5 faster than perfectly written assembly :)

I disagree with a lot of what you’re writing. However, we’ve reached the point where we need to run benchmarks and analyze the generated code (this is easy to do for compiled languages using https://godbolt.org/, but for the JVM, it can be a bit more complex, given the warm-up factor).

So, there is one fundamental point I started with:

> JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null)

And your answer is:

> A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do.

Essentially, you are saying that the compiler can apply aggressive optimizations when it knows what is happening in the code.

But I say that JIT is needed so the compiler can figure out what is happening in the code and perform aggressive optimizations.

There are many things that can be inferred from the code without needing to execute it. The question is how difficult it is to make such an inference: in one scenario, the compiler might attempt to track whether specific data changes-and, if it can prove this, mark the data as immutable and apply certain optimizations-whereas in another, it might already possess the information that the data is immutable.

Moreover, information about immutability is useful not only to the compiler but also to the programmer. Just like information about types: it benefits both the compiler and the programmer. Imagine a fan of JS or Python joining our conversation and claiming that both Java and Rust are low-level languages because you have to specify types - something they view as complex and a hindrance to development speed.

The same applies to the GC: the compiler can perform more optimizations when it knows when memory needs to be cleared (move it to stack or even place the data on registers). The JVM attempts to do this (via escape analysis), but there are limitations; consequently, data ends up on the heap, and GC operations come at a cost (due to data movement).

Rust simply makes it easy to obtain far more information, enabling aggressive optimizations that are both immediate and guaranteed.

There remain a small number of cases, such as `switch` statements - where one branch executes 99% of the time, while the other 99 branches execute only 1% of the time. In such instances, the JIT could indeed perform further optimizations; however, I am not even sure if the overhead of monitoring wouldn't outweigh the benefits. And the question is when and how to perform PGO, or whether to perform it at all.

Yes, and the important point is that when it comes to knowing things statically, abstraction and optimisation are in conflict. The whole point of abstraction is that the implementation details aren't known. So in C++ we always suffer from this problem called "zero overhead abstractions" or "abstraction costs", which means that to give the compiler the information it needs, we have to use less general abstractions, which are viral and harm evolution. What a JIT does is allow the compiler to learn the very things that abstraction hides; yes, it's a virtual call, yes, it could target anything, but I've seen it hit the same target 1000 out of the last 1000 times, so I speculate that this will continue and I'll inline even though I could be wrong.

> The same applies to the GC: the compiler can perform more optimizations when it knows when memory needs to be cleared

I understand why this could be true in theory, but in practice the problem is:

1. not that the compiler knows when an object is unreachable, but that the generated code has to do something at that point, and

2. the most efficient known memory management algorithms - moving collectors and arenas, both work in nearly the same way - are entirely predicated on freeing memory in bulk and on not doing anything when an object becomes unreachable, and so the knowledge of when an object becomes unreachable doesn't help them.

So it is true that C and C++ and Rust always statically know when an object is dead, and you could say that hypothetically they don't need to do anything with that information, but in practice they all act on that information immediately and that's inefficient.

> There remain a small number of cases, such as `switch` statements - where one branch executes 99% of the time, while the other 99 branches execute only 1% of the time.

So the main practical benefit of a JIT isn't that at all, but that it can do the "mother of all optimisations" - inlining - far more aggressively. Inlining is important because it cracks open the abstraction boundary of the inlined subroutine, and allows the compiler to further specialise and optimise things, now with the appropriate context.

Anyway, all of these fundamental questions and differences between languages with more statically known information and figuring out "unprovable" information in practice were very well known before the JVM was built to address the performance problems we had suffered from in large C++ programs. So we can argue over which workloads are helped by this and which aren't, but there is no way to say which is usually faster in the absract (because, again, these considerations were known and taken into account). It's merely an empirical question, and not one that's easy to settle. After more than 25 years of working with C++ and almost 20 years of working with Java, my default is that low-level wins on performance (if written by experts) in smaller programs, and Java wins on performance in larger programs, but of course, there are many caveats in either direction.

I disagree with that phrasing; it is better to say that abstractions allow a programmer to ignore unimportant details. For example, when developing two modules (possibly even by different teams), all they know about each other is a lean interface, without any implementation details. However, the compiler might know everything.

> So in C++ we always suffer from this problem called "zero overhead abstractions" or "abstraction costs"

This is another odd term. In Rust, the term used instead is "zero-cost abstractions," referring to cases where the compiler can generate instructions for higher-level code just as efficiently.

> So the main practical benefit of a JIT isn't that at all, but that it can do the "mother of all optimisations" - inlining - far more aggressively.

I’ll reiterate that I disagree with this: inlining is performed very efficiently during monomorphization. And monomorphization is used very frequently in Rust.

> After more than 25 years of working with C++

I don't have much experience with C++; I mostly use Rust. I can only assume that the C++ development experience is far worse than Rust - especially when trying to write software that is both reliable and fast. This may be particularly relevant to older C++.

So, a Dog, a Cat, and an Abstract Mammal walk into a bar...

https://godbolt.org/z/dEsW1sfM8

I didn't want to do this, but I went ahead and created a small example showing that monomorphization and inlining work remarkably well. (Obviously, this example does not address memory management)

You're talking about abstraction at the code level; I'm talking about abstraction at the language level. A virtual call means "the implementation is unknowable here", and it is, indeed, rarely knowable to an AOT compiler.

> This is another odd term. In Rust, the term used instead is "zero-cost abstractions," referring to cases where the compiler can generate instructions for higher-level code just as efficiently.

Rust took that term from C++ (and it had slightly different ones over the years). What it means that the language offers different mechanisms - chosen statically - with different abstraction levels (i.e. different generality) and different costs, some of which are zero, but often similar or identical-looking code at the use site, because the mechanism choice depends on some non-local information, typically associated with the type. I call it "writes like a low-level language, reads like a high-level one". This is different from C (or Zig), which usually makes the selected mechanism explicit at the use site, or from Java, which chooses the cheapest applicable mechanism at every use-site for a single general construct.

The problem is that, because the mechanims is chosen statically, you need to choose the cheapest applicable mechanism, usually virally, yourself, and that over time this gets harder or things drift toward the more general and costly mechanisms.

That's what Java tried to solve, but there are, of course, tradeoffs. The obvious one (which has solutions) is warmup time, because the compiler needs to wait to learn what optimisations can be applied even if they're unprovable, e.g. to learn that a polymorphic application is actually monomorphic in practice at a particular call-site (the solution is to cache the optimised machine code from one run to the next). The more fundamental tradeoffs are 1. you're not guaranteed which mechanism will be chosen, 2. there can be a bad, though amortised, worst-case due to deoptimisation (this is what happens when the compiler optimises too aggressively and then finds out it was wrong, e.g. it inlined a virtual call under the assumption it's the only target at the use site, but after a while, another target appears (in Rust/C++, you'll always pay the higher price, but there's no point at which deoptimisation occurs), and 3. you need an FFI layer, as you can't take the machine address of a compiled subroutine (as it may be re-compiled multiple times).

Tradeoffs 2 and 3 are the main reasons low-level languages don't do this optimisation, and 3 is particularly important. Low-level languages are designed, first and foremost, to be low level. To do its sophisticated optimisations, Java needs to move around pointers to both code and data, which requires a clear FFI layer between Java code and anything external. Having such an FFI layer in a low-level language (and I'm not talking about Rust/C++'s thin extern FFI) defeats the very purpose of a low-level language, which is to talk directly to the hardware and OS. That is the chief goal of all low-level languages, and they sacrifice everything for it. Not only safety (Rust's unsafe is used relatively pervasively) but also performance.

> I can only assume that the C++ development experience is far worse than Rust

Actually, the experience in the two languages is remarkably similar, and not by accident. Rust certainly improves some details, but the overall experience "in the large" is very close. But note that the performance problem is not because of "zero cost abstractions" but because of the low-levelness and focus on the worst-case. Even in Zig, which tries hard to avoid zero cost abstractions to keep use sites explicit, the choice between a specific-and-cheap and a general-and-expensive mechanism means that for best performance you need to pick a less general mechanism, and that gets trickier and trickier to maintain as the program evolves over the years, and especially if it's large.

> monomorphization and inlining work remarkably well.

Of course it does, which is why the optimising JIT was invented: to make it work more broadly!

This wasn't done just on principle, but to solve a very real problem. What we used to do in C++ is architect a solution and write code that monomorphises in all the right places - because that's what one does - and the result was good and fast. And then, five years later, we had to add some feature and were faced with the choice of either undoing some core optimisation or re-architecting some 10,000 LOC. The problems didn't arise when first writing the program, when everything was known. It arose when some change - that hadn't been foreseen when the program was first written - had to be done. Java didn't make the first step substantially cheaper; it made all the following work - five, ten, fifteen years down the line - substantially cheaper.

An important caveat is that HotSpot currently misses many auto-specialisation opportunities that it could take advantage of, but that's one of the things that make working on such a cutting-edge compiler so interesting :) The problem, as always, isn't just the work required, but also determining which optimisations actually make a difference in real programs (and not just in specific benchmarks).

Of course, now there's this hypothesis that AI could do this costly rearchitecting for you, even in large programs, but it doesn't do it well (at all!) today, and I think that when we get to a point where it can do it well, it will also be smart enough to do it in machine code directly (or at least in C), at which point all programming languages will be over. What I don't think is likely is that AI will be able to do extremely complex semantics-preserving large-scale transformations correctly, yet still need the help of a sophisticated compiler for much more local transformations and far simpler correctness checks.

Caffeine's next release has roughly 25% higher read throughput, with unchanged write throughput, thanks to fixing a false sharing mistake. That won't be visible in real workloads, but is fun nonetheless (500M reads/s on 8 cores).

I was reading your comments on Java, nodding my head, upvoting, without checking your profile and realizing that you're a member of the Java team. Knowing that doesn't mean I now suddenly disagree with you or anything. But while in an ideal world it doesn't matter who's saying something when evaluating it, there's some human factors at play - I'd like to turn up my internal sense of skepticism when dealing with someone, effectively, selling something their salary depends on; even if you're being entirely earnest, it's ultimately a sales pitch, and I feel bamboozled for not recognizing it - that'd make me appreciate transparency.

(FWIW, even though I prefer being coy about my place-of-work, I have no professional relation to this conversation. I've never used Java in my 9-5 and I haven't even really used it in earnest since, like, version 5 back in high school. I think it's always been underrated by the hacker crowd, though!)

This is really, really silly. Java is many times beyond the position where its developers need to desperately convince people to use it. This is a person who has unique technical expertise in the area whose credentials are smack dab on their profile, not hidden from you. Their closeness to the domain at hand should make you less skeptical of what they are saying.

I also don't see the case for stability. Yes, if you're still on JDK 8, it would probably chug on for a couple of years. But we were talking about greenfield projects and newer JDK go EOL much faster. If you want patches, you'll have to run your app to a newer JDK, which may break a couple of things. Rust (within the same edition) or Go (within the same major version) break less than that.

As far as runtime compatibility goes, Rust and Go apps ship with the runtime. This can be better or worse for you, depending on what is your upgrade story, but I don't see a clear winner here. What I would give to Java over Rust is that you will have far fewer dependencies to take care of if you need to upgrade. But the same goes for Go.

For observability, I feel that with Rust you have a bit less that you need to observe (no GC to worry about). Tokio tracing is great, but observability requires a bit more effort. The go observability story is far worse. So Java probably has an edge here, but not something that ever felt like a game changer. My impression is that for most of the enterprise shops that love Java, observability means collecting unstructured log files through NFS and trying to find a needle in the haystack with primitive tools, but I've been out of touch with this world for a couple of years.

Productivity is something that is dead if you are AI-heavy. Sure, many shops are still wary about AI, and I totally get why, but this is a battle that's already been lost. Without AI, I would say I was about 3 to 4 times more productive in Rust than I was in Java, but ramping up that productivity took at least 1 year of practice. It's not time most companies are willing to spend. With AI, this doesn't matter anymore, for better or worse.

I'm not arguing that Java is not chosen often for greenfield projects. It's clearly extremely popular in many circles, especially outside startups and big tech. But I think the reason Java is chosen have little to do with the reasons you've mentioned above and more with organizational preferences.

Java also breaks very few things. Breaking binary compatibility is a no-go since it's a core promise of the platform. The only thing in the surface language that has ever been changed is the meaning of the underscore as an identifier, as well as the behavior of == in upcoming Project Valhalla.

> As far as runtime compatibility goes, Rust and Go apps ship with the runtime.

Java applications can also be shipped together with the runtime.

> Productivity is something that is dead if you are AI-heavy.

Nevertheless, making constructs available to express intent more clearly should also help LLMs to not go off the rails.

It isn't, and the problem isn't Rust specifically, but all low-level languages. They can offer very good performance (often better than Java) when small. But as they evolve over time, or are very large to begin with, they become much harder to keep performant. This is for pretty fundamental constraints of low-level language that I mention in another comment here, and this performance problem with large programs written in low-level languages was well known before Java even existed. The JVM was designed, at least in part, to address it.

One of the things that drew me to Java (from years of C++, even though I still work in C++ when I work on the JVM) is precisely how it addresses those performance issues we ran into with C++ five years into a project.

> As far as runtime compatibility goes, Rust and Go apps ship with the runtime. This can be better or worse for you, depending on what is your upgrade story, but I don't see a clear winner here.

I wasn't talking about "runtime compatibility" but of overall version compatibility. Java has an unmatched compatibility record - not perfect, but better than anything else (with at least a medium-sized standard library).

> For observability, I feel that with Rust you have a bit less that you need to observe (no GC to worry about).

Memory management is very often a bigger issue without a GC than with a moving GC. Time and again we see Rust or C++ programs spend 30-50% on memory management.

> Productivity is something that is dead if you are AI-heavy.

Really? Have you had AI write a good medium-sized (say 100-500 KLOC) program or maintain one over a long period of time without very close reviews? The only people I've seen who don't know about the ticking time-bomb agents leave in the codebase are the people who don't look.

> With AI, this doesn't matter anymore, for better or worse.

You may be talking about small programs. I agree that for small programs, low-level languages can offer excellent performance, and AI can be okayish, and you can get some observability you can live with, but I'm talking about large programs.

> But I think the reason Java is chosen have little to do with the reasons you've mentioned above and more with organizational preferences.

Those organisational preferences are due to a long record of delivering on the things I mentioned. Java has an exceptionally low "regret factor", i.e. people who regret choosing it five, ten, or fifteen years into a project (which is when the problems usually start).

Generally, efficient memory management is orthogonal to object oriented design. Meaning, as your complexity grows and your business logic changes, it often means the optimal memory management changes because the lifecycle and relationship between objects change.

For a web server for instance, you have both request/response as well as various transactional memory requirements. In Java, the role of the garbage collector is to adapt to whatever the best memory policy is based on runtime behavior, rather than statically defined rules. One could say that the evolutionary and revolutionary changes in garbage collectors as well as the multitude of tuning parameters comes from this being a really hard task.

If you have a services architecture, the runtime advantages of Java go down significantly.

> I wasn't talking about "runtime compatibility" but of overall version compatibility. Java has an unmatched compatibility record.

I would say both matter significantly more again in a monolithic architecture. It matters a lot more when you are trying to deploy your software into a single application server, or trying to avoid version incompatibilities when integrating large amounts of code into a single executable.

> Time and again we see Rust or C++ programs spend 30-50% on memory management.

I've seen plenty of Java applications spend 30 seconds or longer because they had to do a full garbage collection back in the day. I even had one customer who maxed out Java to utilize all the memory in their server and hit a 13 minute production pause due to otherwise unoptimized GC (promoting many temporary transactional objects to the mature generation until it eventually exhausted memory).

The different strategies for memory management (static vs dynamic) ultimately still require recognizing, diagnosing and correcting issues. GC provides unique challenges because the tuning mechanism is decoupled from the actual code. GC challenges can also often go undiagnosed until staging/production workloads hit them, precisely because they are dynamic behaviors.

30-50% of what?

Right, so one case (which I certainly believe is possible) is very different from “time and time again.”

As for the caching test, it's just technically interesting, because the JVM was designed to address the performance issues we suffered from in large C++ programs (all the JVM engineers are, of course, C++ people), both due to compilation and to memory management, and we regularly compare both our compilation and memory management algorithms to other approaches, and it just so happens that last week one of our GC engineers compared Caffeine to Moka and saw how CPU-intensive the memory management work is compared to ZGC (he was particularly interested in this because caching is one of the more challenging workloads for generational moving GCs because a cache deals with many old objects, whereas generational GCs tend to focus more on young objects, and he wanted to make sure that our GCs help reduce the high memory-management overheads associated with low-level languages even in this challenging scenario).

I'm not sure what kind of tools you mean, but unless you're looking for something that just works exactly the way EJBs do for some mysterious reasons, I don't see why you can't do most "enterprisey" things with Rust or Go. Or Python or TypeScript for that matter.

Yes and that's exactly what modern tooling is missing. Try to develop for node.js 0.2.12 on today's update of Visual Studio Code. See? No enterprise-level collaboration for ya.

Exactly. Never updated, never discussed, never challenged. Set in stone.

It felt like every dev that worked on our Java behemoth at a previous job was elated to switch to Go.

Rust is too low-level for typical enterprise app where requirements changes twice a day. You end up spending time and tokens fighting with borrow checker.

C# is MS product, which is no-go for some folks.

Kotlin probably would be the answer.

Which Java famously does not have.

> Rust is too low-level for typical enterprise app where requirements changes twice a day. You end up spending time and tokens fighting with borrow checker.

In my experience, you do not spend tokens fighting with the borrow checker anymore, newer models are smarter. But it might not be ideal for a lot of CRUD applications.

> C# is MS product, which is no-go for some folks.

This is 2026, it's not 1996 anymore. .Net works on Linux and Microsoft is as friendly towards open source and open standards as a Big Tech company can be.

If anything, it was Oracle which more recently sued another company for using a JDK alternative. And this was a lawsuit that, if accepted, could have put the entire idea of API compatibility in danger and deal a severe blow to the Open Source movement.

Anyone who is morally bothered by MS but is unfazed by this is probably just mentally stuck in the 1990s.

> Kotlin probably would be the answer.

I love Kotlin, but I'm afraid that's not the case. The conservative organizations that choose Java out of inertia, would keep choosing Java over Kotlin, even if Kotlin is a better JVM language which is facing no downside.

For anyone who doesn't need to be on the JVM or work with JVM tooling, Kotlin doesn't cut it. It doesn't have null pointer dereference problem in theory... Only it does in practice if you're using any Java API that may return null (all these bang-decorated "Platform types"). Generic type erasure can only be overcome in inline functions with reified types. And building and deploying artifacts without docker is still a mess.

I found Kotlin extremely publishing for Java shops in the past, and I've converted multiple departments totaling over hundreds of employees to use Kotlin. But that was before AI. The rationale was simple: Java is an entrenched language that leads to bloated code, slow development cycles and way too many avoidable bugs in productions. Kotlin solves some if these issues, and it's very easy to learn for a Java engineer, while still letting you keep all of your tools and libraries. And as a language (putting ecosystem aside), I find it better than either Go or Typescript, and far more ergonomic than Rust[1].

But all of these arguments die with AI. Rust is just as ergonomic as any other popular language today if you're using an agent, and the fact that an engineer spent their lifetime writing Spring Boot programs in Java you don't have time to let them learn a new stack from scratch doesn't matter anymore.

Sure, there are many companies where letting AI write the code is still not acceptable, but most of these workplaces will accept AI agents sooner than they accept Kotlin.

I feel a bit sad since I like many ideas about Kotlin (especially how amenable it is for making DSLs) but we've lost that opportunity

--

[1] Unless you have to write highly concurrent code without any data races.

Have you worked on large (>500KLOC) codebases with an agent? Not only do you have to be an expert at the language, but even if you're lucky and everything is fine, Java code is likely to be particularly fast by comparison, because the agents aren't very good at manual optimisation, especially as the code grows (they're even worse than humans at that, and humans aren't great at manual optimisation of large codebases, either, which is one of the problems the JVM set out to solve; in fact, agent-written code in a low-level language gets pretty slow well below that size). Oh, and the long build times certainly don't help.

Yes. But keep in mind KLOCs are not easily comparable across languages. Java is notoriously verbose. A 500KLOC codebase in Java would usually be half that size in Rust. If your argument is that large codebases makes life harder for agents, you should go with a less verbose language.

I'm not sure what "manual optimization" means (isn't it a bit of an oxymoron when the agent does it?), but if your agent has the proper tools (e.g. ast-grep, rg, semble) it can deal with large codebases. Would the agent create slop? Yes. But it wouldn't be worse on the slop that humans created on every moderately-sized Java project I've worked on.

> in fact, agent-written code in a low-level language gets pretty slow well below that size

I've never seen this happening. I've seen agents writing suboptimal Rust code (e.g. copies instead of Cow). But while this occassionally happens with Rust, I've never seen an agent optimizing for Java where necessary (e.g. using object pools to avoid GC churn). Java is not magic.

Lol, no way. Especially that rust is pretty verbose all things together (which makes sense, given it's a low level language - ergo you have to literally express more things about the code)

That isn't really a comparison of the languages as much as the standard runtimes and ecosystems. It is important to consider that each have comparable components.

So you aren't comparing a no_std rust project against a comparable JavaCard, but say Diesel vs Hybernate code examples around ORM.

Meanwhile in java it's a constant "Arc<JavaType>", and scopes don't mark "drop points"

Ok, so you barely know either Java or Rust.

> If your argument is that large codebases makes life harder for agents, you should go with a less verbose language.

You mean, like Rust??? But no, that's not my argument. Agents have a hard time keeping up the architecture in large software (and the differences between verbose languages like Java, Go, Rust, and C++ vs less verbose ones like Python and JS don't make much of a difference). So they either make a mess or they do the simple thing, and the simple thing in low-level languages is often slow.

> But it wouldn't be worse on the slop that humans created on every moderately-sized Java project I've worked on.

Yeah, I don't think you've actually tried it.

> I've never seen an agent optimizing for Java where necessary (e.g. using object pools to avoid GC churn). Java is not magic.

Object pools are far less efficient than Java's GCs, but while state-of-the-art compiler and memory management technology is certainly not magic, I suggest you learn more about these things if you want to make informed decisions.

Hmm? Java gets null dereferences all the time, that's what a NPE is. The VM takes on the extra plumbing to surface a dereference of a null pointer in a recoverable way to code. On Windows this is done using SEH, on Unix it is handling SIGFAULT - but each NPE corresponds to a null pointer dereference that java then tries to clean up.

That the language does not have a way to have compiler enforced "never null" is actually a huge productivity drain, specifically because you have to do your own defensive measures against null or attempt cleanup/recovery when it happens.

Even languages like Swift which use Optional (e.g. a maybe monad) to provide a concept of nilability still internally will hit null pointer dereferences on occasion with faulty bridged code/bindings. However, they treat this as a non-recoverable violation of invariants - a developer shouldn't be trying to recover from incorrect code at runtime.

if you gonna work in a big team or need a project with lots of devs then yeah go for the JVM.

but if you're doing things on the smaller / small scale side. - just use JS/TS or python. you benefit from cheap runtimes such as Cloudflare workers.

Banks, telcos, etc. aren’t monoliths either. They use plenty of different languages depending on the team, system, and requirements. Java isn’t inherently the choice for greenfield software just because reliability matters.

And of the 4 non-preview JSRs in the Java 27 release, only 1 of them is actually part of the "platform version".

The other 3 are strictly changes to Hotspot internals with no platform involvement at all. They did not change any aspect of any Java platform in any way whatsoever. That is what I'm referring to. I'm not referring to the fact that the core library, language syntax, and runtime specs are all part of the same version. I'm referring to the fact that Hotspot specific behaviors and adjustments are also branded as being part of the platform release.

Like there's no Java 27 platform spec that says that G1 is the default garbage collector. That would of course be an absurd platform spec change. But that is still somehow a "feature" of the Java 27 release according to Oracle.

BTW, Java is developed "code first", which means that we first work on the implementation in OpenJDK, and then extract the relevant spec changes from it.

> But that is still somehow a "feature" of the Java 27 release according to Oracle.

It's a feature of the OpenJDK JDK, which is, indeed, the Java implementation done by Oracle (with contributions from others). The language is very careful, as you can see in the announcement: "JDK 27, the reference implementation of Java 27". The Java SE 27 spec is here: https://www.jcp.org/en/jsr/detail?id=402

I'd read it.

Applications that don't upgrade their runtime are usually not in the market for new libraries (or even new features in old libraries).

Why would anyone think that having significantly less than 50% women requires no investigation if they didn't think women are intrinsically bad for a company? The problem isn't a skew in favour of women. If you expect 50-50 and get 30-70, obviously that's something to look into, but you're complaining there aren't people trying to increase the skew further. It's like complaining about the lack of organisations explicitly calling to raise the growth in average temperature as a counterweight to all those who want to lower it, or about the lack of charities promoting deforestation.

1. Upgrade your JDK for the best performance (as the article says, the slowdown is gone in JDK 26).

2. Don't try to help the GC by pooling objects. Mutating old objects can be expensive, while allocating new ones is cheap (at least for objects that don't do some exceptionally expensive initialisation).

There are also middle ground options, like pooling objects but giving the pool a lifecycle that is tied to a request.

In particular, the JDK's GCs are heavily optimised for short-lived objects with high allocation rates. What you want to avoid is temporary data finding itself in the old gen. The newer GCs may dynamically size the young generation to match your program's natural meaning of "short lived", but the longer an object lives, the higher the chances it ends up in the old gen. You want only objects that stick around for a very long time (and have a low allocation rate) to end up in old gen. If you just write naive code, chances are things will work out well. Once you start being clever, you're taking a risk.

So if you're willing to profile your program, with a workload that's representative of production workload (a microbenchmark is useless) on every runtime version and potentially change your "manual optimisation" every six months, you can try. But if not, the advice for the best performance over time is to rely on the platform and let it do its thing. The reason is that the JVM is continuously being optimised for "normal programs". If you're doing anything too clever, you may find that in a future release, your code is making things worse because the optimisations that target normal code don't help your code (or could even treat it as unusual and have it hit slow paths).

I once spoke to a company that were very proud in getting something like a 10% improvement over naive code in Java 8, thanks to some hand optimisation they worked a lot on, only to discover that it caused a 15% regression compared to doing nothing special on JDK 11.

Removing allocation pressure can also have effects on other parts of the system, but that is anything but trivial to measure or reason about.

The parallel collector is a perfectly fine collector, particularly for smaller heaps. Even the serial collector isn't bad for things like a containerized environment, yet G1 replaces it by default now [1].

It's not a bad algorithm, but especially when you start talking about sub 2G environments I've not seen a situation where the parallel and serial collectors won't handily beat G1 on pretty much every metric. Major collectors with modern CPUs just doesn't take much time for a lot of memory.

[1] https://openjdk.org/jeps/523

I just have a problem with G1 because in my experience, the best place for it is fairly large heaps. Get something sub 2 or even 10G, especially if you have a few cores to offer, and the parallel and often even the serial collector will give G1 latency even on major collections with superior throughput and overhead.

In the modern enterprise Java world (that I've been exposed to) it's very common to have a mandate that all components deploy a minimum of N instances across X regions for resiliency. By design that almost always means you're deploying at least 2x more compute than you strictly need, so the top priority is generally minimizing per-instance overhead to minimize cloud spend.

For example, the default templates at my current company deploy something like 0.25-0.5 vCPU per instance, and therin lies the rub. ZGC performance is _catastrophically_ bad with <=1 cpus because when there's only one core, any "concurrent" GC events become full on stop the world events. We had someone pilot a change to the default JVM args for all components because they heard that ZGC would reduce latency, only to discover that basically all of our microservices immediately failed their perf tests. For the first one I spot checked, throughput was down ~90% and p95 went from ~40ms to >1s, because more time was being spent on "background" GC than actually servicing requests.

Hope that didn't come off adversarial. I just find GC fascinating, and ended up spending a bunch of time working with the team that owns those defaults to draft general recommendations. TLDR is that when in doubt don't specify/let the JVM pick for you, and don't be surprised if it picks serial :)

That throughput penalty is not very high with generational ZGC. It's not zero, but it's not very high, either. What ZGC mostly does is spread the memory management activity more evenly across the duration of the program (this does have a cost due to barriers being active more, but it's not huge). But we have some work planned to improve ZGC even further, which is why I didn't say I think it will become the default imminently, only eventually.

> ZGC performance is _catastrophically_ bad with <=1 cpus

That may well be true, but the JVM can automatically choose a different default algorithm for these circumstances. Indeed, until very recently, the default for low-CPU environments was different (Serial) than for bigger ones (G1).

So right now, when we want control, we need to give up some safety, but weaker things are still helpful.

Also, in low-level code, the problem of "I might forget to do something" sometimes clashes with the problem of "I need to see exactly what operations are done and where". Various kinds of implicitness help with the former at the expense of the latter.

I'm not saying this is universally better than other approaches, but many people who do serious low-level programming would prefer this.

This is a very, very, very common claim. And unfortunately I have no other way to describe it other than a strawman.

In 95% (at least) of the application that need systems programming (not to talk about all applications that don't necessarily need it but will benefit from the performance and it wasn't an option because C++ wasn't an option), you have at most 20% (wildly overestimating) of code that needs to be unsafe. The rest could be completely safe. And amongst code that must be unsafe, you can very commonly encapsulate it in some safe pattern. Many times even extract it to a reusable crate.

That is the point of Rust. Not avoiding unsafety, but limiting and encapsulating it. And evidence proves that to work (for example https://blog.google/security/rust-in-android-move-fast-fix-t...).

Obviously, but that doesn't help me if the complexity in the unsafe parts is made worse, while the safety helps the parts where little help is needed. It's not like the danger in a C program is spread evenly, either.

> but will benefit from the performance

Not so much. Safe Rust is faster than Python and Go for sure, but is, on average, about as fast as Java and C#; sometimes faster, sometimes slower.

So what you're really getting is, typically, a smaller executable, a faster startup, and lower footprint (which is actually a much more complicated matter, but I won't get into it now) in exchange for significantly higher evolution and maintenance costs forever. This is a good and reasonable tradeoff for small programs, especially CLI tools, and not a very good tradeoff for larger and/or longer-lived programs.

That's not the deal I'm getting on either side of this.

> In all benchmarks I saw

If you trust those benchmarks then you deserve whatever you pick. I was talking about experienced experts who understand performance. Low-level languages can help your performance when the program is small and they generally hurt it when it grows large, evolves through many people etc. This is something that people with a lot of experience in low-level languages know.

Appeal to (an unnamed) authority? I consider myself an experienced experts who understands performance and this also matches my experience. While you often can reach the same level of performance in Java or C#, it involves horribly unidiomatic code, unlike in Rust (or C++).

In general, low-level programming languages yield relatively fast small programs, but relatively slow large programs, and with Java/C# it's generally the opposite. The low-level control that helps performance when you're small, starts hurting it when you're big.

> This is why most large and long-running programs have abandoned low-level programming languages.

That's not true, as evidenced by the fact that this move has started before extremely sophisticated JIT compilers or garbage collectors were invented. The reason was not because managed languages were faster or even had equal speed, but because of the costs associated with memory unsafety (not just security), exactly what Rust prevents (which was of course not available then).

You can see empirical evidence of this, for example, by the post about Aurora DSQL rewrite in Rust (https://www.allthingsdistributed.com/2025/05/just-make-it-sc...). One notable quote:

> But after a few weeks, it compiled and the results surprised us. The code was 10x faster than our carefully tuned Kotlin implementation – despite no attempt to make it faster. To put this in perspective, we had spent years incrementally improving the Kotlin version from 2,000 to 3,000 transactions per second (TPS). The Rust version, written by Java developers who were new to the language, clocked 30,000 TPS.

You also ignore the impact of memory usage, where unmanaged language have an even greater edge (yes I know it is possible to optimize managed languages' memory consumption as well. Not to the same amount and often at the expense of speed).

I don't know how long you've been programming, but that's not true. In the late nineties and early aughts I was working on large, performance-critical, soft- and hard-realtime systems, and we only started moving away from C++ when Java started beating its performance.

> The reason was not because managed languages were faster or even had equal speed, but because of the costs associated with memory unsafety (not just security), exactly what Rust prevents (which was of course not available then).

That's a myth, and a fairly recent one. Sure, there were non-performance-sensitive programs written in slow languages for a long time. But the industry was mostly using C++ for anything that needed to be big and fast, and back then "memory safety" was mostly just another type of bug. It was nowhere near reason enough to use slow languages, which is why we didn't use them.

Lack of memory safety is a serious problem, but the claim that it's the biggest issue with C++, let alone the one that's always been considered the biggest issue, is just a myth. Back then it was certainly considered no bigger an issue than the language complexity, compilation time, and even performance issues in large, long-running programs.

> You can see empirical evidence of this, for example, by the post about Aurora DSQL rewrite in Rust

I talk to the people at AWS, and this is not the evidence you think it is. First, their problem was primarily with GC pauses, and it was before pauseless GCs. Second, the codebase isn't very big. Third, because Java and C++/Rust offer similar performance - sometimes one wins, sometimes another - you expect to see exactly that. I can tell you that we recently wrote a distributed cache in both Java and Rust simultaneously (using the pauseless GC). The Java version achieved twice the throughput of the Rust version, and significanly better latency across all percentiles. So sure, on the smaller end, there are programs where Rust would be 2x as fast as Java, there are programs where Java would be 2x as fast as Rust, and on average they're about the same. But over time, Java's advantage starts to show as it makes it easier to keep the good performance over years of evolution.

Yeah, this is not true, and there's no time limit. I mean, maybe some JIT compilers, like JavaScript's have a time limit, but their goal is to run JS at an acceptable speed. Java's JIT is intended to reduce the runtime overheads of AOT compilers, and the only way to do that is by optimising significantly more than AOT compilers, obviously not less (otherwise, we'd just always use an AOT compiler).

You can easily see why there's no time limit if you understood how Java's optimising JIT works. First, code is run in the interpreter and some profiles are collected, then a non-optimising JIT runs and continues to collect profile, and finally the optimising JIT runs. The vast majority of the time is spent waiting for profiles to collect, and so if compilation itself runs, say, even 3x slower, it won't even be perceptible. Also, because we have profiles, we don't have to compile much of the program at all, because we know what the hot spots are. Initialisation code that runs once is never compiled (remember, the focus is long-running programs, exactly those that low-level languages have trouble with).

Finally, the reason sophisticated JIT compilers can optimise more - which is why they're used in the first place - is thanks to speculative optimisation. AOT compilers need to spend a lot of time on optimisation, and even then they are limited, because they need to prove that the program transformation is valid (i.e. that there's no miscompilation). The power of JIT compilers is that they don't. They only need to speculate that a certain profile will continue to be in effect. So if so far some virtual call always hits a certain target, they can go ahead and inline it (not only to the cost of a regular call, but to no call at all, and then they optimise the whole inlined code). If they're wrong, a fault triggers and they decompile the relevant subroutine going back to the interpreter and non-optimising compiler.

> and that C++ and Rust are allocating much, much less than Java and even C# or Go, so a faster allocation scheme is much less needed there

This is true, but the causaility here is that the reason we avoid allocation in C++ is precisely because it's so slow.

> but even in those cases it's usually possible to alleviate the costs with wise organization of allocations

The problem is that this is true in principle. In practice this is certainly true in smaller programs. In larger programs, this work is not easy at all, and you find yourself doing harder and harder work just to keep up.

> including using arenas etc. in some places

One of the reasons I'm excited about Zig (I'm a low-level programmer) is that it makes arenas much more viable. Arenas in C++ and especially Rust are not really a pleasure to work with, and they're viral and a constant maintenance burden. BTW, the reason moving GCs are so fast is that they work quite similarly to arenas.

> and they're also offloaded from the better-optimizing compiler.

It's a worse-optimising compiler. In C++, I use templates to achieve similar optimisation to what Java does, and in Zig I can use comptime, and again, it's certainly possible but it's hard work. You can't let the templates explode all over the codebase, and, as it evolves, you have to go back and profile and take out the ones that no longer help, replacing them with new ones.

Just to tell you a bit about me, I was a C++ programmer for many years, and when Java showed up, like many, I was sceptical. When I saw that the JIT + moving collector hypothesis actually accomplishes its goal in reducing the overheads we were seeing in C++ in many situations, I went to work on the JVM. Back then there were still latency tradeoffs due to GC pauses, but GC pauses no longer exist as of three years ago.

Now, a lot of people, including some of the world's top compiler and memory management experts, believe that the vision of using JITs and moving collectors to address the performance problem of low-level languages is working exceedingly well. We can certainly argue about under which conditions Java wins and under which C++ (or Zig it Rust etc.) win and how common they are, but people who think low-level languages win across the board or almost across the board clearly don't know what's going on. Early on it was people who were sceptical about how effectively JITs and moving collectors could do their job in practice (even though the theory was clear), but these days I think it's mostly people who haven't struggled with performance issues in low-level languages long enough, and just see that for small or young programs they work fine. They always were. Writing a new program in C++ was never harder than writing a new program in Java, and the performance was great (and people weren't concerned about memory safety in particular). The problems came later - in the 5th year, the 10th year, etc.., when the cost of evolution and trying to keep performance good were piling up.

Not to mention the availability of advanced tooling like MIRI.

Second, the reason why the compiler cannot infer on its own that a class does or does not need identity without you declaring it is that the use of identity can be in a different module.