One aspect of virtual threads that has gone somewhat unappreciated so far is that if you (ab)use some internal APIs you can probably implement the testing technique used by FoundationDB.
For those who don't know, FoundationDB is a horizontally scalable transactional (KV) database used by Apple for their cloud services. It's famous primarily for its testing approach in which they can run the entire set of microservices in a simulator that can both inject faults, but also completely control the scheduling of work. In this way they make the system entirely deterministic, and can then pseudo-randomize simulations to uncover race conditions and the like.
To do this they created a custom DSL on top of C++ called Flow which is compiled to coroutines.
Flow and what it enables is considered (by them at least) to be a key competitive advantage over other database engines.
With Loom the JVM now happens to support the very same thing, just not officially. That's because the JVM makes file and network IO fully "pluggable", virtual threads can be supplied with a custom scheduler, and classloaders can be used to run multiple programs within the same JVM (with some caveats). So if you wanted to make a deterministic simulation of your microservice based system then you could do that now.
The above would be a rather advanced project for intrepid developers only, but it'd be interesting to see someone explore that at some point.
It was explored a bit in a blog article with some sample code. One big issue with this approach right now is that the scheduler is not officially user replaceable. It's possible using reflection to do so though, at least is was in 2022, the internal code may have changed since then. There's been some indication from the OpenJDK team that this might be an officially supported feature in the future.
Cassandra has a deterministic testing framework in this vein that will support Loom before too long. Loom is more of an efficiency enhancement, however, permitting you to run a simulation with a single operating system thread, avoiding the overhead of managing which thread the OS is permitted to run. The framework already intercepts all of these operations and ensures they are executed deterministically.
Sure. That is not the sort of thing that is guaranteed to catch all bugs. Model checking approaches are better at providing proofs of non-trivial properties.
The problem is that model checking checks a model which often fails to properly match the actual implementation. Plenty of people have used model proven algorithms but had implementation bugs that mean that the highly desired properties proven for the model are not upheld.
Being able to automatically check the implementation against specific race conditions (etc.) helps to provide confidence in the implementation working correctly, which means your model proven results are more likely correct for the actual implementation.
Excellent! With virtual threads, all the blocking code I wrote is efficient now :)
Less-joking: I'm so excited for this to start getting adoption. For a pre-release/preview feature, Loom already has SO much traction, with most major frameworks already having support for running app/user code on virtual threads and one notable project (Helidon Nima) replacing netty with their own solution based on blocking IO and virtual threads. Now I want to see the community run with it.
I've always thought async IO was just plain gross.
Python's implementation is so yucky that after using it for one project I decided that I'd rather DIY it with multiprocessing than use async again. (I don't have any more constructive feedback than that, my apologies, it was a while ago so I don't remember the specifics but what has lasted is the sour taste of the sum of all the problems we had with it - perhaps notably that only about 2 people on my dev team of 5 actually understood the async paradigm).
netty did it fine. I've built multiple projects on top of netty and it's fine. I like event-based-async more than async-await-based-async. But it's still a headache and notably I really rather missed the kinds of guarantees you can get (in blocking code) by wrapping the block in try-catch-finally (to e.g. guarantee resources get freed or that two counters, say a requests-in and a requests-out, are guaranteed to match up).
But dang am I excited to not do that anymore. I have one specific project that I'm planning to port from async to blocking+virtualthreads that I expect to greatly simplify the code. It has a lot of requests it makes back and forth (it has to manually resolve DNS queries among other things) so there's good chunks of 50-200 ms where I have to either yield (and has gross async code that yields and resumes all the heck over the place) or block the thread for human-noticeable chunks of time (also very gross of course!).
Funny I think async in Python is a lot of fun for side projects but my experience is that if I hand my Python systems off to other people they usually have trouble deploying them and invariably can’t maintain them.
Whereas my Java projects live on long after I am gone from the project.
Personally I love aiohttp web servers, particularly when using web sockets and brokering events from message queues and stuff like that. Not to mention doing cool stuff with coroutines and hacking the event queue (like what do you do if your GUI framework also has an event queue?) If YOShInOn (my smart RSS reader + intelligent agent) were going to become open source though I might just need to switch to Flask which would be less fun.
Async in Python, for a long time, has been a horrible hack relying on monkey patching the socket module. The newer asyncio stuff is quite nice by comparison, but the problem is that Python, due to its popularity, has libraries that haven't been upgraded.
Python always had deployment issues, IMO. In Java, 99% of all library dependencies are pure JARs, and you rarely need to depend on native libraries. You can also assemble an executable fat JAR which will work everywhere, and the fact that the build tools are better (e.g., Maven, Gradle) helps.
Compared with Python, for which even accessing a RDBMS was an exercise in frustration, requiring installing the right blobs and library headers via the OS's package manager, with Postgres being particularly painful. NOTE: I haven't deployed anything serious built with Python in a while, maybe things are better now, but it couldn't get much better, IMO.
Fully agree on Helidon Nima and blocking IO. Zero hope that Spring framework crapola will not smother already massively simplified thing with convenient abstractions on top it.
Can you explain why you're excited about with virtual threads? I get that they improve throughput in extremely high pressure apps, but the JVM's current threading isn't exactly a slouch. Java's used in HFT shops, and more generally in fintech where performance matters.
The main problem is that it's not a matter of "speed" but just of congestion.
If you write a program using blocking IO and Platform (OS) threads, you're essentially limited to a couple hundred concurrent tasks, or however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering. So it's slow not because Java is slow, but because kernel threads are heavyweight and you can't just make a trillion of them just for them to be blocking waiting on IO.
If you use async approaches, your programming model suffers, but now you're multiplexing millions of tasks over a small number of platform threads of execution still without even straining the kernel's switching. You've essentially moved from kernel scheduling to user-mode scheduling by writing async code.
Virtual threads is a response to this, saying "what if you can eat your cake and have it, too?" by "simply" providing a user-mode-scheduled thread implementation. They took the general strategy that async programming was employing and "hoisted" it up a couple levels of abstraction, to be "behind" the threading model. Now you have all the benefits of just blocking the thread without all the problems that come from trying to have a ton of platform threads that will choke the linux kernel out.
I can’t speak for the OP, but this makes it much easier to write code that uses threads that wait on IO, and just let the underlying system (VM + JDBC connectors, for example) handle the dirty work.
A few years ago, I wrote a load generation application using Kotlin’s coroutines - in this case, each coroutine would be a “device”. And I could add interesting modeling on each device; I easily ran 250k simulated devices within a single process, and it took me a couple of days. But coroutines are not totally simple; any method that might call IO needs to be made “coroutine aware”. So the abstraction kinda leaks all over the place.
Now, you can do the same thing in Java. Just simply model each device as its own Runnable and poof, you can spin up a million of them. And there isn’t much existing code that has to be rewritten. Pretty slick.
So this isn’t really a “high performance computing” feature, but a “blue collar coder” thing.
Pre-v21, Java's threads were 1:1 based on OS threads. HFT apps were normally single-threaded with exotic work-dispatching frameworks, native-memory buffers &| built-in network stacks so, while being undoubtedly fast, were not particularly representative.
V21 virtual threads are more like Go's goroutines. They map 1:m with OS threads, and the JVM is responsible for scheduling them, making them much less of a burden on the underlying OS, with fewer context switches, etc. And the best thing is, there has been minimal change in the Java standard library API, making them very accessible to existing devs and their codebases.
An interesting part of HFT is you normally do everything on a single thread. Your unit of parallelism would be a different JVM and you want to avoid context switching at all costs, going as far as to pin specific OS threads, making sure your thread for execution is never used by GC.
To understand the benefit of virtual threads, I think it's helpful to think of it as a "best of both worlds" situation between blocking IO and async IO. In summary, virtual threads give you the scalability (not simply "performance") benefits of async IO code while keeping the simplified developer experience of normal threads and blocking IO.
First, it's best to understand the benefit of virtual threads from a webserver. Usually, a webserver maps 1 request to 1 thread. However, most of the time the webserver actually doesn't run much code itself: it calls out to make DB requests, pulls files from disk, makes remote API requests, etc. With blocking IO, when a thread makes one of these remote calls, it just sits there and waits for the remote call to return. In the meantime, it holds on to a bunch of resources (e.g. memory) while it's sitting doing nothing. For something like HFT, that's normally not much of a problem because the goal isn't to server tons of independent incoming requests (sometimes, obviously the usage pattern can differ), but for a webserver, it can have a huge limiting effect on the number of concurrent requests that can be processed, hurting scalability.
Compare that to how NodeJS processes incoming web requests. With Node (and JS in general), there is just a single thread that processes incoming requests. However, with async IO in Node (which is really just syntactic sugar around promises and generators), when a request calls out to something like a DB, it doesn't block. Instead, the thread is then free to handle another incoming web request. When the original DB request returns, the underlying engine in Node essentially starts up that request from where it left off (if you want more info just search for "Node event loop"). Folks found that in real world scenarios that Node can actually scale extremely well to the number of incoming request, because lots of webserver code is essentially waiting around for remote IO requests to complete.
However, there are a couple of downsides to the async IO approach:
1. In Node, the main event loop is single threaded. So if you want to do some work that is heavily CPU intensive, until you make an IO call, the Node server isn't free to handle another incoming request. You can test this out with a busy wait loop in a Node request handler. If you have that loop run for, say, 10 seconds, then no other incoming requests can be dispatched for 10 seconds. In other words, Node doesn't allow for preemptive interruption.
2. While I generally like the async IO style of programming and I find it easy to reason about, some folks don't like it. In particular, it creates a "function coloring" problem: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... . Async functions can basically only be called from other async functions if you want to do something with the return value.
Virtual threads then basically can provide the best features from both of these approaches:
1. From a programming perspective, it "feels" pretty much like you're just writing normal, blocking IO code. However, under the covers, when you make a remote call, the Java schedule will reuse that thread to do other useful work while the remote call is executing. Thus, you get greatly increased scalability for this type of code.
2. You don't have to worry about the function coloring problem. A "synchronous" function can call out to a remote function, and it doesn't need to change anything about its own function signature.
3. Virtual threads can be preemptively interrupted by the underlying scheduler, preventing a misbehaving piece of code from starving resources (I'm actually less sure of the details on this piece for Java).
Yes, while green and red threads were an implementation detail not exposed to Java programmers, now both thread models are exposed (Threads and Virtual Threads).
Additionally since virtual threads are exposed across the whole runtime and standard library, not only they are built on top of native threads (red), the developers have control over their scheduling.
The TLDR is that it needs “function coloring” which isn't necessarily bad, types themselves are “colors”, the problem being what you're trying to accomplish. In an FP language, it's good to have functions that are marked with an IO context because there the issue is the management of side effects. OTOH, the differences between blocking and non-blocking functions is: (1) irrelevant if you're going to `await` on those non-blocking functions or (2) error-prone if you use those non-blocking functions without `await`. Kotlin's syntax for coroutines, for example, doesn't require `await`, as all calls are (semantically) blocking by default. We should need extra effort to execute things asynchronously.
One issue with “function coloring” is that when a function changes its color, all downstream consumers have to change color too. This is actually useful when you're tracking side effects, but rather a burden when you're just tracking non-blocking code. To make matters worse, for side-effectful (void) functions, the compiler won't even warn you that the calls are now “fire and forget” instead of blocking, so refactorings are error-prone.
In other words, .NET does function coloring for the wrong reasons and the `await` syntax is bad.
Furthermore, .NET doesn't have a usable interruption model. Java's interruption model is error-prone, but it's more usable than that. This means that the “structured concurrency” paradigm can be implemented in Java, much like how it was implemented in Kotlin (currently in preview).
PS: the .NET devs actually did an experiment with virtual threads. Here are their conclusions (TLDR virtual threads are nice, but they won't add virtual threads due to async/await being too established):
You're supposed to either await a Task, or to block on it (thus blocking the underlying OS thread which probably eats a couple of megabytes of RAM). It's a completely different system more akin to what Go has been using.
Yes, but this is supposedly transparent. At least until you interface directly with native libraries, or with leaky abstractions that don’t account for that.
Sadly, no-one has managed to package it yet, but we should get something in the next couple of days. Since 21 is an "LTS" release, major Linux distributions will provide a runtime pretty soon. Ubuntu backports them to old releases too.
Just to be clear, all of these sites offer builds of Oracle software (the OpenJDK JDK), developed by Oracle and licensed by Oracle. When you file an issue that's in the JDK, those companies forward it to us, Oracle engineers, to fix.
Java downloaded from Oracle has the following license:
> Important Oracle Java License Information
> The Oracle Technology Network License Agreement for Oracle Java SE is substantially different from prior Oracle Java licenses. This license permits certain uses, such as personal use and development use, at no cost -- but other uses authorized under prior Oracle Java licenses may no longer be available. Please review the terms carefully before downloading and using this product.
Getting JDK from better sources won't risk you accidentally hitting the wrong download link and winding up with an enterprise licensing audit and a legal requirement to Java license for every employee.
I'd strongly recommend you don't download anything Java related from Oracle or have any direct relationship with Oracle at all. Java employees working for Oracle and their advice should be treated skeptically. Reporting issues to third parties and letting them contact Oracle's engineers is ideal (least of all because you need a support contract to report it directly to Oracle anyway).
Oracle is a bad actor and should be avoided for your own good. The only ones suggesting getting it from Oracle directly are also Oracle employees.
Oracle is responsible for 95+% of every single commit going to OpenJDK, which has the exact same license and situation as the goddamn linux kernel, GPL2. Do you also sweat a lot about accidentally installing red hat linux? If not, you have zero reason to worry.
How is amazon compiling and repackaging the exact same thing somehow different?
People have been burned too many times by Oracle “true-ups”. And no, it’s not FUD. Their license explicitly states that JEE/SE (not JRE) is covered by their commercial license, not their open source license. This year, in 2023, Oracle introduced the enterprise-wide metric license model. So if you use Java for anything in your organization, they charge you based on your organization size. (All or nothing model).
There is cognitive overhead in even having to consider how Oracle could be licensing anything you download from them. Their licensing has a long history of being complex, expensive, and enforced. Choosing to get software from a less complicated (licensing wise) provider is a valid choice.
The licensing risk of using open source software, that is mostly being written by Oracle, from Amazon is less.
Amazon's open source reputation is that they'll screw over maintainers with their rebranded forks, but not go after users of open source.
Oracle's OpenJDK builds are only updated for six months after each release, while Adoptium receives long-term support (like the commercial Oracle JDK).
Oracle provides roughly 3/4 of the commits and fixed issues. It's a majority, but there are other significant players (always surprised to see SAP higher then Google and Amazon).
I get what you mean, but given answers here (and other places in this thread) it seems like the licensing is not clear at all.
At least it's not clear to me, I'm still on "Oracle is evil, go as far as you possibly can from them". I would genuinely be happy to be able to confidently say that "this particular project of Oracle is safe for me to use", though.
Yeah the lack of anyone packaging this up was kinda weird to me. I understand these are done on someone else's dime, and no one owes me a free lunch though.
I don't really pay a lot of attention to Java releases but I was actually excited about this one. No one ran a build process last night in anticipation?
Oracle's stance on what contexts their JDK/JRE is free to use in is ... nuanced, and has changed in the past. While I don't wish to spread FUD, the thought of using it in a commercial environment in any fashion would give me pause. Playing around with Oracle on licensing is, as hobbies go, somewhere on my list of things to do down around making plutonium cocktails and alligator juggling.
The other vendors have different (typically more explicitly open) licensing on their shipped artifacts. As ever, if it really matters, it's time to consult a legal professional in your jurisdiction to be sure instead of taking a random internet nerd's opinion on it. ;)
There are two main branches of JDKs, OpenJDK and Oracle JDK (both maintained by Oracle).
Builds for the former are open-source but only updated for 6 months after each release, while the latter is under a commercial license with 3+ years commercial long-term support for some versions.
Projects like Eclipse Temurin (Adoptium) bring the LTS to OpenJDK. The underlying code is entirely open-source, so there is no licensing trouble.
Hence, if you want to download OpenJDK, it's important to do it from Adoptium, as you won't get updates on java.net after a while.
As much as I want to clown a little on the language for taking so long to do string templates, congratulations on getting virtual threads shipped, I can't imagine the effort that must've took to implement into something as hairy and complex as the JVM!
I have a genuine question about Loom / virtual threads / etc.
In languages with async-await syntax, I like that it is explicit what is sync and what is async. I also like how the "bind" points within an async expression make it clear where the syncronization happens. It seems to me that Java 21 makes all of this implicit, which is a step backward in this respect.
Similar to how types enforce valid usage (a foundational idea in Java), I feel that an Async<T> should not be implicitly a T.
Ideally knowing "what is synchronous and what is asynchronous" shouldn't be your concern, just like freeing memory isn't your concern in a language with garbage collection. Similarly, you don't have to know the "real" memory addresses when using virtual memory.
Ask yourself why do you want to know what is asynchronous. In a typical server-side app synchronous, blocking, thread-per-task code is easy to write and read. There is only one drawback: you can run out of kernel threads. Asynchronous code solves this problem, but virtual threads solve it in a better way, because you can have three orders of magnitude more virtual threads compared to kernel threads. Ultimately, it's all about the number of threads you are allowed to use.
Async vs green threads doesn't alter the number of running threads available to you, it's simply a difference in how the runtime schedules your virtual threads on the limited number of real threads. Async/await is cooperatively scheduled, Java's virtual threads are preemptively scheduled.
Both can have similar performance characteristics, the difference is primarily one of ergonomics, and since that's subjective there are people who will argue fiercely for either side.
I agree with you—I prefer explicit synchronization. But this isn't an accidental flaw in Java's design, it's the entire purpose of their chosen model. A lot of people dislike the fact that async/await causes function coloring [0] and as a result may lead to duplication of sync and async version of functions. These people see Java as dodging a bullet and learning from past mistakes in other languages, so it's unlikely that Java will have anything comparable to async/await.
Yes. The preemptive style (where scheduling points are determined dynamically) is more composable than the cooperative style (where scheduling points are determined statically). In the cooperative style, the correctness of any subroutine is implicitly dependent on there not being scheduling points where they're not explicit; adding a scheduling point to any subroutine, therefore, requires reconsidering the assumptions in all of its transitive callers. In the preemptive style, on the other hand, any subroutine can independently express its mutual-exclusion assumptions using various synchronisation constructs, such as locks (those independent decisions could result in deadlocks, but the cooperative style is also subject to deadlocks).
So if you start from scratch, the preemptive style is preferable, but for us the decision was even easier, because we weren't starting from scratch. Java already had threads, and so mutual exclusion was already explicit rather than implicit.
When is cooperative scheduling more appropriate? In a language like JavaScript that already had a lot of existing code with implicit mutual exclusion assumptions (because JS didn't have threads). To add concurrency to such a language with a large existing ecosystem, cooperative scheduling is a better choice.
> So if you start from scratch, the preemptive style is preferable
I don't believe that you proved that in the preceding paragraph. All you said is that cooperative scheduling requires making the scheduling points explicit while preemptive doesn't, but "explicit is better than implicit" is a principle that a substantial portion of the programming community believes, and OP specifically expressed a preference for explicit scheduling points.
For myself, I also want to see scheduling points propagate up the call tree the same way that I want checked exceptions. The lack of composability is a feature, not a bug. I prefer to know which parts of my code are most likely to be problematic and then keep those parts isolated from the rest.
I recognize that this is a subjective opinion and not everyone believes what I do, and for some use cases preemptive is better, but I don't see why you need to pretend that you made the most optimal decision instead of just acknowledging that it's complicated and you made the one that you feel is best for Java.
> In the cooperative style, the correctness of any subroutine is implicitly dependent on there not being scheduling points where they're not explicit
Removing double negations, you're saying "cooperative style is bad because the runtime can't interrupt user code implicitly"? You are looking at this from the perspective of the runtime implementer (meaning yourself, thank you by the way), not the programmer. You absolutely should look at this from the perspective of the programmer too.
Preemptive multitasking gives the runtime more capabilities because you can just assume the user has to deal with anything that your implicit rescheduling causes. But users don't actually deal with that, even if they should! The havoc that concurrency bugs are causing is evidence for that!
JavaScript's concurrency model is so nice because despite lacking real parallelism, you only have to think about concurrency (race conditions, data races, deadlocks, ...) at scheduling points. Rust has an interesting model with both parallelism and cooperative async-await. Its type system requires strong guarantees to move data between (preemptively scheduled) threads, but much weaker guarantees to use it in a cooperatively scheduled async function. Of course, it comes at a cost, namely function coloring.
After decades of legacy, none of that is feasible in Java of course, but let's not pretend that preemptive scheduling is objectively "preferable".
In many languages the distinction is there because you have two versions of the IO routines, synchronous and asynchronous, and calling a synchronous one will block you asynchronous task. With virtual threads we took a different approach: if you make an IO call on a virtual thread it will do the IO asynchronously if at all possible and yield execution. This means you can take existing code and run it on virtual threads without modification and it will pretty much just work.
Not all IO operations can be done asynchronously on all platforms, and some new interfaces had to be introduced to allow for asynchronous name lookup, and some code needs refactoring because it had implicit assumptions that it would be used by tasks in a small thread pool and so used thread local values as a cache for expensive objects, but in general I think the design has achieved the goal of allowing existing code to be moved to virtual threads without the requirement to rewrite everything or scatter await / async all over the place.
It's a different priority, not a down grade. Most people in Loom discussions really don't understand why cooperative async is set up the way it does and why explicit yielding is important if you need to manage a single UI or GPU bound thread. Most of the replies you get are just thinking about raw bandwidth of nameless threads.
But the structured concurrency stuff they added is sorta kinda familiar to explicit yields (as long as you set up everything properly). IMO its uglier than async/await but I also deal with main threads a lot.
I'm excited to see if this breeds any interesting UI frameworks or if its mostly just about server bandwidth.
The question of "single UI or GPU bound thread" could be solved in a future release with custom virtual thread schedulers. Java 21 concentrates on solving the problem of server-side applications.
This is an old discussion, Unyielding (https://glyph.twistedmatrix.com/2014/02/unyielding.html) discusses the trade-offs between function coloring and explicit transaction control from the point of view of "I need explicit transaction control" and of course What Color Is Your Function (https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...) discusses it from the point of view of function coloring. Which one you want depends on your application, unfortunately.
Structured concurrency does give you a “structure” on possible fork and join points, for a more high-level management.
Within virt threads, does it really matter? Every function takes as much time as it takes - the reason you use virtual threads is to make that efficient, the semantics is clear - your code is blocking, and waits for every previous statement.
Also, it is not quite valid to model it as types - these are more like effects.
I'm not personally concerned, no. You need to think about threads to efficiently exploit multi-core hardware anyway. Otherwise the best you can do is hope you're implementing isolated requests with multi-processing, and assume that you'll never benefit to exploit intra-operation parallelism.
One problem with async/await is that the term "blocking" is under-specified. If you try and nail down what is and isn't blocking, like with a formal way to classify operations as such, you rapidly end up with a lot of contradictions and gray areas. It's a distinction that doesn't really make sense except as a hack to work around expensive threads/callstacks, which is what Loom solves.
For example, is reading a file blocking? On modern computers this could do anything spanning from being purely a set of branches and pointer dereferences as a kernel block cache hit, thus being little different to any other code, or it could be an NVMe flash request (extremely fast, probably not worth switching the thread), or it could be an HDD seek (fairly slow, worth freeing up the thread to do something else), or it could be a network RPC (extremeley slow, definitely worth freeing up the thread). You may not even know ahead of time what it is. So, should the API be blocking? Or marked async?
The right place to figure this out isn't in hand-written code. That'd be a performance hack similar to the old C register keyword. The runtime has much more information on which to make the right call, but for that, the code has to be written in a uniform style.
More generally, type systems ideally aren't encoding performance-specific stuff into them. Type systems help with program organization and correctness, and as a happy side effect can also guide compilers to make better code, but the performance of an operation shouldn't itself be exposed in its prototype. Otherwise optimizations can break code, which wouldn't make sense, being able to optimize modules independently is a big part of why you want dynamic linking.
If you design your software right, you can use async/await similarly to IO in Haskell—functions that are not async are fast and unlikely to fail, while functions that are async are potentially slow and likely to throw exceptions. This partition can help you reason about and structure your code as you're writing it for solid error handling and minimal duplicate I/O.
A common pattern is to use a single thread executor as one context. Any task scheduled to that context will not run concurrently. When you do it this way, every synchronous chunk (between yields/awaits) is a critical section. So you get critical sections easily, cheaply and with a pretty good syntax.
This pattern is used a lot when you have things like UI which use a single threaded design, or if you need to bind to do some IPC in a single native thread. It's actually very common.
Some of the recommendations in the list date back to .NET Framework.
Most of the time, it is not something one needs to worry about because actually problematic cases are highlighted by default Roslyn analyzers and will give you a warning on build as well.
Async/await is a superior model that is more flexible and solves more tasks (heh) at hand than green threads which have more limited scope of application, higher base overhead and cannot reach parity at efficiency due to stack state having to be always preserved.
That is a conscious design choice by the language designers. They feel that having to "color" your code as either async or sync is tedious leads to code duplication. After having worked a lot with Kotlin coroutines, I tend to agree.
Virtual threads don't give you async IO, they give you (the illusion of) more threads. If one VT blocks, the underlying OS thread switches to another VT. All IO is blocking.
Finally virtual threads I can use in production with generational ZGC! I've been using Kotlin coroutines for a while now, and want to see how that evolves with JVM now supporting virtual threads out of box.
Both - Kotlin's coroutines are a runtime for green threading. The implementation of that runtime could use Java's green threads for many things and where the developer UX or guarantees aren't there, fall back on other primitives. But users of Kotlin could also just use Java's new virtual threads if they don't need anything fancy from Kotlin's coroutine runtime.
You probably wouldn't want to _mix_ these threading approaches in a single codebase unless Kotlin's coroutines become a literal "dev UX facade" because otherwise things like ThreadLocals (and whatever Kotlin provides as an alternative) can get out of sync.
Oracle managed to create one of the least ergonomic string interpolation syntaxes of any language I've ever seen. Using backslashes is truly a terrible move
C# $"{x} plus {y} equals {x + y}"
Visual Basic $"{x} plus {y} equals {x + y}"
Python f"{x} plus {y} equals {x + y}"
Scala s"$x plus $y equals ${x + y}"
Groovy "$x plus $y equals ${x + y}"
Kotlin "$x plus $y equals ${x + y}"
JavaScript `${x} plus ${y} equals ${x + y}`
Ruby "#{x} plus #{y} equals #{x + y}"
Swift "\(x) plus \(y) equals \(x + y)"
Is any of those actually better? You have a non-existing better suggestion that works in the same cases they outline in the JEP?
I think the following is the main part to keep in mind:
> For Java, we would like to have a string composition feature that achieves the clarity of interpolation but achieves a safer result out-of-the-box, perhaps trading off a small amount of convenience to gain a large amount of safety.
It's fine to break backwards compatibility when introducing an entirely new feature across a major version, actually. What's even the point of a major version bump?
May I introduce to Swift? They even decided on that without any backwards compatibility limitations. It really is not bad, does it really matter if it’s a $ or a \ ?
using backslashes for anything other than escaping a characters (e.g: \" or \n) makes it a nightmare to try to coerce the compiler into understanding what you're trying to do if you combine escape characters and syntax in the same string
Readit News
For those who don't know, FoundationDB is a horizontally scalable transactional (KV) database used by Apple for their cloud services. It's famous primarily for its testing approach in which they can run the entire set of microservices in a simulator that can both inject faults, but also completely control the scheduling of work. In this way they make the system entirely deterministic, and can then pseudo-randomize simulations to uncover race conditions and the like.
To do this they created a custom DSL on top of C++ called Flow which is compiled to coroutines.
https://www.youtube.com/watch?v=4fFDFbi3toc
Flow and what it enables is considered (by them at least) to be a key competitive advantage over other database engines.
With Loom the JVM now happens to support the very same thing, just not officially. That's because the JVM makes file and network IO fully "pluggable", virtual threads can be supplied with a custom scheduler, and classloaders can be used to run multiple programs within the same JVM (with some caveats). So if you wanted to make a deterministic simulation of your microservice based system then you could do that now.
The above would be a rather advanced project for intrepid developers only, but it'd be interesting to see someone explore that at some point.
Blog article: https://jbaker.io/2022/05/09/project-loom-for-distributed-sy... HN discussion: https://news.ycombinator.com/item?id=31314006
Deleted Comment
The problem is that model checking checks a model which often fails to properly match the actual implementation. Plenty of people have used model proven algorithms but had implementation bugs that mean that the highly desired properties proven for the model are not upheld.
Being able to automatically check the implementation against specific race conditions (etc.) helps to provide confidence in the implementation working correctly, which means your model proven results are more likely correct for the actual implementation.
Less-joking: I'm so excited for this to start getting adoption. For a pre-release/preview feature, Loom already has SO much traction, with most major frameworks already having support for running app/user code on virtual threads and one notable project (Helidon Nima) replacing netty with their own solution based on blocking IO and virtual threads. Now I want to see the community run with it.
I've always thought async IO was just plain gross.
Python's implementation is so yucky that after using it for one project I decided that I'd rather DIY it with multiprocessing than use async again. (I don't have any more constructive feedback than that, my apologies, it was a while ago so I don't remember the specifics but what has lasted is the sour taste of the sum of all the problems we had with it - perhaps notably that only about 2 people on my dev team of 5 actually understood the async paradigm).
netty did it fine. I've built multiple projects on top of netty and it's fine. I like event-based-async more than async-await-based-async. But it's still a headache and notably I really rather missed the kinds of guarantees you can get (in blocking code) by wrapping the block in try-catch-finally (to e.g. guarantee resources get freed or that two counters, say a requests-in and a requests-out, are guaranteed to match up).
But dang am I excited to not do that anymore. I have one specific project that I'm planning to port from async to blocking+virtualthreads that I expect to greatly simplify the code. It has a lot of requests it makes back and forth (it has to manually resolve DNS queries among other things) so there's good chunks of 50-200 ms where I have to either yield (and has gross async code that yields and resumes all the heck over the place) or block the thread for human-noticeable chunks of time (also very gross of course!).
Whereas my Java projects live on long after I am gone from the project.
Personally I love aiohttp web servers, particularly when using web sockets and brokering events from message queues and stuff like that. Not to mention doing cool stuff with coroutines and hacking the event queue (like what do you do if your GUI framework also has an event queue?) If YOShInOn (my smart RSS reader + intelligent agent) were going to become open source though I might just need to switch to Flask which would be less fun.
Python always had deployment issues, IMO. In Java, 99% of all library dependencies are pure JARs, and you rarely need to depend on native libraries. You can also assemble an executable fat JAR which will work everywhere, and the fact that the build tools are better (e.g., Maven, Gradle) helps.
Compared with Python, for which even accessing a RDBMS was an exercise in frustration, requiring installing the right blobs and library headers via the OS's package manager, with Postgres being particularly painful. NOTE: I haven't deployed anything serious built with Python in a while, maybe things are better now, but it couldn't get much better, IMO.
If you write a program using blocking IO and Platform (OS) threads, you're essentially limited to a couple hundred concurrent tasks, or however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering. So it's slow not because Java is slow, but because kernel threads are heavyweight and you can't just make a trillion of them just for them to be blocking waiting on IO.
If you use async approaches, your programming model suffers, but now you're multiplexing millions of tasks over a small number of platform threads of execution still without even straining the kernel's switching. You've essentially moved from kernel scheduling to user-mode scheduling by writing async code.
Virtual threads is a response to this, saying "what if you can eat your cake and have it, too?" by "simply" providing a user-mode-scheduled thread implementation. They took the general strategy that async programming was employing and "hoisted" it up a couple levels of abstraction, to be "behind" the threading model. Now you have all the benefits of just blocking the thread without all the problems that come from trying to have a ton of platform threads that will choke the linux kernel out.
A few years ago, I wrote a load generation application using Kotlin’s coroutines - in this case, each coroutine would be a “device”. And I could add interesting modeling on each device; I easily ran 250k simulated devices within a single process, and it took me a couple of days. But coroutines are not totally simple; any method that might call IO needs to be made “coroutine aware”. So the abstraction kinda leaks all over the place.
Now, you can do the same thing in Java. Just simply model each device as its own Runnable and poof, you can spin up a million of them. And there isn’t much existing code that has to be rewritten. Pretty slick.
So this isn’t really a “high performance computing” feature, but a “blue collar coder” thing.
V21 virtual threads are more like Go's goroutines. They map 1:m with OS threads, and the JVM is responsible for scheduling them, making them much less of a burden on the underlying OS, with fewer context switches, etc. And the best thing is, there has been minimal change in the Java standard library API, making them very accessible to existing devs and their codebases.
First, it's best to understand the benefit of virtual threads from a webserver. Usually, a webserver maps 1 request to 1 thread. However, most of the time the webserver actually doesn't run much code itself: it calls out to make DB requests, pulls files from disk, makes remote API requests, etc. With blocking IO, when a thread makes one of these remote calls, it just sits there and waits for the remote call to return. In the meantime, it holds on to a bunch of resources (e.g. memory) while it's sitting doing nothing. For something like HFT, that's normally not much of a problem because the goal isn't to server tons of independent incoming requests (sometimes, obviously the usage pattern can differ), but for a webserver, it can have a huge limiting effect on the number of concurrent requests that can be processed, hurting scalability.
Compare that to how NodeJS processes incoming web requests. With Node (and JS in general), there is just a single thread that processes incoming requests. However, with async IO in Node (which is really just syntactic sugar around promises and generators), when a request calls out to something like a DB, it doesn't block. Instead, the thread is then free to handle another incoming web request. When the original DB request returns, the underlying engine in Node essentially starts up that request from where it left off (if you want more info just search for "Node event loop"). Folks found that in real world scenarios that Node can actually scale extremely well to the number of incoming request, because lots of webserver code is essentially waiting around for remote IO requests to complete.
However, there are a couple of downsides to the async IO approach:
1. In Node, the main event loop is single threaded. So if you want to do some work that is heavily CPU intensive, until you make an IO call, the Node server isn't free to handle another incoming request. You can test this out with a busy wait loop in a Node request handler. If you have that loop run for, say, 10 seconds, then no other incoming requests can be dispatched for 10 seconds. In other words, Node doesn't allow for preemptive interruption.
2. While I generally like the async IO style of programming and I find it easy to reason about, some folks don't like it. In particular, it creates a "function coloring" problem: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... . Async functions can basically only be called from other async functions if you want to do something with the return value.
Virtual threads then basically can provide the best features from both of these approaches:
1. From a programming perspective, it "feels" pretty much like you're just writing normal, blocking IO code. However, under the covers, when you make a remote call, the Java schedule will reuse that thread to do other useful work while the remote call is executing. Thus, you get greatly increased scalability for this type of code.
2. You don't have to worry about the function coloring problem. A "synchronous" function can call out to a remote function, and it doesn't need to change anything about its own function signature.
3. Virtual threads can be preemptively interrupted by the underlying scheduler, preventing a misbehaving piece of code from starving resources (I'm actually less sure of the details on this piece for Java).
Hope that helps!
Additionally since virtual threads are exposed across the whole runtime and standard library, not only they are built on top of native threads (red), the developers have control over their scheduling.
Virtual threads make blocking IO calls automagically non-blocking, allowing for better utilization of the CPU.
The TLDR is that it needs “function coloring” which isn't necessarily bad, types themselves are “colors”, the problem being what you're trying to accomplish. In an FP language, it's good to have functions that are marked with an IO context because there the issue is the management of side effects. OTOH, the differences between blocking and non-blocking functions is: (1) irrelevant if you're going to `await` on those non-blocking functions or (2) error-prone if you use those non-blocking functions without `await`. Kotlin's syntax for coroutines, for example, doesn't require `await`, as all calls are (semantically) blocking by default. We should need extra effort to execute things asynchronously.
One issue with “function coloring” is that when a function changes its color, all downstream consumers have to change color too. This is actually useful when you're tracking side effects, but rather a burden when you're just tracking non-blocking code. To make matters worse, for side-effectful (void) functions, the compiler won't even warn you that the calls are now “fire and forget” instead of blocking, so refactorings are error-prone.
In other words, .NET does function coloring for the wrong reasons and the `await` syntax is bad.
Furthermore, .NET doesn't have a usable interruption model. Java's interruption model is error-prone, but it's more usable than that. This means that the “structured concurrency” paradigm can be implemented in Java, much like how it was implemented in Kotlin (currently in preview).
PS: the .NET devs actually did an experiment with virtual threads. Here are their conclusions (TLDR virtual threads are nice, but they won't add virtual threads due to async/await being too established):
https://github.com/dotnet/runtimelab/issues/2398
https://adoptium.net
https://aws.amazon.com/corretto
https://www.azul.com/downloads
https://bell-sw.com/pages/downloads
Sadly, no-one has managed to package it yet, but we should get something in the next couple of days. Since 21 is an "LTS" release, major Linux distributions will provide a runtime pretty soon. Ubuntu backports them to old releases too.
We offer our own builds under two different licenses on our own websites (https://jdk.java.net or https://www.oracle.com/java/technologies/downloads/) but you can also download our software on the other websites linked above.
> Important Oracle Java License Information
> The Oracle Technology Network License Agreement for Oracle Java SE is substantially different from prior Oracle Java licenses. This license permits certain uses, such as personal use and development use, at no cost -- but other uses authorized under prior Oracle Java licenses may no longer be available. Please review the terms carefully before downloading and using this product.
[0] https://www.java.com/en/download/
Getting JDK from better sources won't risk you accidentally hitting the wrong download link and winding up with an enterprise licensing audit and a legal requirement to Java license for every employee.
I'd strongly recommend you don't download anything Java related from Oracle or have any direct relationship with Oracle at all. Java employees working for Oracle and their advice should be treated skeptically. Reporting issues to third parties and letting them contact Oracle's engineers is ideal (least of all because you need a support contract to report it directly to Oracle anyway).
Oracle is a bad actor and should be avoided for your own good. The only ones suggesting getting it from Oracle directly are also Oracle employees.
Deleted Comment
Oracle is responsible for 95+% of every single commit going to OpenJDK, which has the exact same license and situation as the goddamn linux kernel, GPL2. Do you also sweat a lot about accidentally installing red hat linux? If not, you have zero reason to worry.
How is amazon compiling and repackaging the exact same thing somehow different?
The licensing risk of using open source software, that is mostly being written by Oracle, from Amazon is less.
Amazon's open source reputation is that they'll screw over maintainers with their rebranded forks, but not go after users of open source.
https://inside.java/2023/09/19/the-arrival-of-java-21/
Oracle provides roughly 3/4 of the commits and fixed issues. It's a majority, but there are other significant players (always surprised to see SAP higher then Google and Amazon).
At least it's not clear to me, I'm still on "Oracle is evil, go as far as you possibly can from them". I would genuinely be happy to be able to confidently say that "this particular project of Oracle is safe for me to use", though.
I don't really pay a lot of attention to Java releases but I was actually excited about this one. No one ran a build process last night in anticipation?
The other vendors have different (typically more explicitly open) licensing on their shipped artifacts. As ever, if it really matters, it's time to consult a legal professional in your jurisdiction to be sure instead of taking a random internet nerd's opinion on it. ;)
Builds for the former are open-source but only updated for 6 months after each release, while the latter is under a commercial license with 3+ years commercial long-term support for some versions.
Projects like Eclipse Temurin (Adoptium) bring the LTS to OpenJDK. The underlying code is entirely open-source, so there is no licensing trouble.
Hence, if you want to download OpenJDK, it's important to do it from Adoptium, as you won't get updates on java.net after a while.
One can optionally buy support for it, that’s the whole deal. Not that hard, but apparently this is a minefield for some..
Java 21 makes me like Java again - https://news.ycombinator.com/item?id=37538333 - Sept 2023 (728 comments)
Java 21: First Release Candidate - https://news.ycombinator.com/item?id=37126530 - Aug 2023 (107 comments)
Java 21: What’s New? - https://news.ycombinator.com/item?id=37067491 - Aug 2023 (256 comments)
Java 21: No more public static void main - https://news.ycombinator.com/item?id=36189923 - June 2023 (77 comments)
JEP 430: String Templates (Preview) Proposed to Target Java 21 - https://news.ycombinator.com/item?id=35012862 - March 2023 (237 comments)
In languages with async-await syntax, I like that it is explicit what is sync and what is async. I also like how the "bind" points within an async expression make it clear where the syncronization happens. It seems to me that Java 21 makes all of this implicit, which is a step backward in this respect.
Similar to how types enforce valid usage (a foundational idea in Java), I feel that an Async<T> should not be implicitly a T.
Are people concerned about this at all?
Have the Java designers addressed this?
Ask yourself why do you want to know what is asynchronous. In a typical server-side app synchronous, blocking, thread-per-task code is easy to write and read. There is only one drawback: you can run out of kernel threads. Asynchronous code solves this problem, but virtual threads solve it in a better way, because you can have three orders of magnitude more virtual threads compared to kernel threads. Ultimately, it's all about the number of threads you are allowed to use.
Both can have similar performance characteristics, the difference is primarily one of ergonomics, and since that's subjective there are people who will argue fiercely for either side.
[0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
So if you start from scratch, the preemptive style is preferable, but for us the decision was even easier, because we weren't starting from scratch. Java already had threads, and so mutual exclusion was already explicit rather than implicit.
When is cooperative scheduling more appropriate? In a language like JavaScript that already had a lot of existing code with implicit mutual exclusion assumptions (because JS didn't have threads). To add concurrency to such a language with a large existing ecosystem, cooperative scheduling is a better choice.
I don't believe that you proved that in the preceding paragraph. All you said is that cooperative scheduling requires making the scheduling points explicit while preemptive doesn't, but "explicit is better than implicit" is a principle that a substantial portion of the programming community believes, and OP specifically expressed a preference for explicit scheduling points.
For myself, I also want to see scheduling points propagate up the call tree the same way that I want checked exceptions. The lack of composability is a feature, not a bug. I prefer to know which parts of my code are most likely to be problematic and then keep those parts isolated from the rest.
I recognize that this is a subjective opinion and not everyone believes what I do, and for some use cases preemptive is better, but I don't see why you need to pretend that you made the most optimal decision instead of just acknowledging that it's complicated and you made the one that you feel is best for Java.
Removing double negations, you're saying "cooperative style is bad because the runtime can't interrupt user code implicitly"? You are looking at this from the perspective of the runtime implementer (meaning yourself, thank you by the way), not the programmer. You absolutely should look at this from the perspective of the programmer too.
Preemptive multitasking gives the runtime more capabilities because you can just assume the user has to deal with anything that your implicit rescheduling causes. But users don't actually deal with that, even if they should! The havoc that concurrency bugs are causing is evidence for that!
JavaScript's concurrency model is so nice because despite lacking real parallelism, you only have to think about concurrency (race conditions, data races, deadlocks, ...) at scheduling points. Rust has an interesting model with both parallelism and cooperative async-await. Its type system requires strong guarantees to move data between (preemptively scheduled) threads, but much weaker guarantees to use it in a cooperatively scheduled async function. Of course, it comes at a cost, namely function coloring.
After decades of legacy, none of that is feasible in Java of course, but let's not pretend that preemptive scheduling is objectively "preferable".
Not all IO operations can be done asynchronously on all platforms, and some new interfaces had to be introduced to allow for asynchronous name lookup, and some code needs refactoring because it had implicit assumptions that it would be used by tasks in a small thread pool and so used thread local values as a cache for expensive objects, but in general I think the design has achieved the goal of allowing existing code to be moved to virtual threads without the requirement to rewrite everything or scatter await / async all over the place.
But the structured concurrency stuff they added is sorta kinda familiar to explicit yields (as long as you set up everything properly). IMO its uglier than async/await but I also deal with main threads a lot.
I'm excited to see if this breeds any interesting UI frameworks or if its mostly just about server bandwidth.
Within virt threads, does it really matter? Every function takes as much time as it takes - the reason you use virtual threads is to make that efficient, the semantics is clear - your code is blocking, and waits for every previous statement.
Also, it is not quite valid to model it as types - these are more like effects.
One problem with async/await is that the term "blocking" is under-specified. If you try and nail down what is and isn't blocking, like with a formal way to classify operations as such, you rapidly end up with a lot of contradictions and gray areas. It's a distinction that doesn't really make sense except as a hack to work around expensive threads/callstacks, which is what Loom solves.
For example, is reading a file blocking? On modern computers this could do anything spanning from being purely a set of branches and pointer dereferences as a kernel block cache hit, thus being little different to any other code, or it could be an NVMe flash request (extremely fast, probably not worth switching the thread), or it could be an HDD seek (fairly slow, worth freeing up the thread to do something else), or it could be a network RPC (extremeley slow, definitely worth freeing up the thread). You may not even know ahead of time what it is. So, should the API be blocking? Or marked async?
The right place to figure this out isn't in hand-written code. That'd be a performance hack similar to the old C register keyword. The runtime has much more information on which to make the right call, but for that, the code has to be written in a uniform style.
More generally, type systems ideally aren't encoding performance-specific stuff into them. Type systems help with program organization and correctness, and as a happy side effect can also guide compilers to make better code, but the performance of an operation shouldn't itself be exposed in its prototype. Otherwise optimizations can break code, which wouldn't make sense, being able to optimize modules independently is a big part of why you want dynamic linking.
https://elizarov.medium.com/how-do-you-color-your-functions-...
If you design your software right, you can use async/await similarly to IO in Haskell—functions that are not async are fast and unlikely to fail, while functions that are async are potentially slow and likely to throw exceptions. This partition can help you reason about and structure your code as you're writing it for solid error handling and minimal duplicate I/O.
This pattern is used a lot when you have things like UI which use a single threaded design, or if you need to bind to do some IPC in a single native thread. It's actually very common.
Most of the time, it is not something one needs to worry about because actually problematic cases are highlighted by default Roslyn analyzers and will give you a warning on build as well.
Async/await is a superior model that is more flexible and solves more tasks (heh) at hand than green threads which have more limited scope of application, higher base overhead and cannot reach parity at efficiency due to stack state having to be always preserved.
You probably wouldn't want to _mix_ these threading approaches in a single codebase unless Kotlin's coroutines become a literal "dev UX facade" because otherwise things like ThreadLocals (and whatever Kotlin provides as an alternative) can get out of sync.
I think the following is the main part to keep in mind:
> For Java, we would like to have a string composition feature that achieves the clarity of interpolation but achieves a safer result out-of-the-box, perhaps trading off a small amount of convenience to gain a large amount of safety.
While I don't like it, all other alternatives were kind of taken by libraries.
And who knows, since it is preview, it might still change.
Deleted Comment