> In comparison with the previous Java service, the updated backend delivers a 40% increase in performance, along with improved scalability, security, and availability.
As is always the case with such rewrites, the big question is whether the improvements came from the choice of language or because they updated a crusty legacy codebase and fixed bugs/bottlenecks.
With the 90% reduction in memory consumption, I'd wager that most if not all the performance improvement came from that. In fact, it is a little surprising that hardware utilization only dropped 50%.
Reduced memory consumption for cloud applications was apparently also the primary reason IBM was interested in Swift for a while. Most cloud applications apparently sit mostly idle most of the time, so the number of clients you can multiplex on single physical host is limited by memory consumption, not CPU throughput.
And Java, with the JIT and the GC, has horrible memory consumption.
IBM is a huge and quite balkanized company and I don't think there was ever a centralized push towards Swift outside some excited parties. With that, I would note that circa 2018 there was a DRAM shortage in the industry and people started thinking more about memory conservation in datacenter workloads.
The thing about DRAM is that it isn't SRAM; cost matters. You struggle to find deployment environments that have less than 1 GB DRAM available per core, because at that point, ~95% the HW cost is typically CPU anyway. Shrinking that further is kind of pointless, so people don't do it. Hence, when utilizing 16 cores, you get at least 16 GB DRAM that comes with it, whether you choose to use it or not. If you use only 10% of that memory by removing the garbage from the heap, then while lower seems better and all that, it's not necessarily any cheaper in actual memory spending, if both fit within the same minimum 1 GB/core shape you can buy anyway. It might just under utilize the memory resources you paid for in a minimum memory per CPU shape, which isn't necessarily a win. Utilizing the memory you bought isn't wasting it.
Each extra GB per core you add to your shape, actually costs something. Hence every GB/core that can be saved results in actual cost savings. But even then, usually every extra GB/core is ~5% of the CPU cost. Hence, even when going from 10 GB/core (sort of a lot) to 1 GB/core, that only translates to ballpark ~50% less HW cost. Since they did not mention how many cores these instances have, it's hard to know what GB/core were used before and after, and hence whether there were any real cost savings in memory at all, and if so what the relative memory cost savings might have been compared to CPU cost.
Interesting. If IBM was trying to solve for memory consumption, why do you think they picked Swift over alternatives that might also have achieved lower memory consumption?
Given that they also experienced a 90% reduction in Memory Usage (presumably from Java GC vs Swift AOT memory management) - it seems more likely the gains are in fact from the difference in languages.
The JVM tends to use as much memory as it can for performance reasons. It is not a reliable indicator of how much memory it actually needs. Why spend resources on clearing memory if there's still unused memory left?
If memory is an issue, you can set a limit and the JVM will probably still work fine
The typical rule of thumb is that getting good performance out of tracing GC requires doubling your memory usage, so a 90% reduction suggests that they made significant improvements on top of the language switch.
The java runtime is a beast. Even the fact that another runtime is just capable to do a similar thing is impressive, disregard the fact that it might be better. Even being on par makes it interesting for me to maybe try it on my own.
You can learn a lot when writing V2 of a thing though. You've got lots of real world experience about what worked and what didn't work with the previous design, so lots of opportunity for making data structures that suit the problem more closely and so forth.
But did they write the backend from scratch or was it based on a number of “com.apple.libs.backend-core…” that tend to bring in repeating logic and facilities they have in all their servers? Or was it a PoC they promoted to MVP and now they’re taking time to rewrite “properly” with support for whatever features are coming next?
Yes! The post would have been much more informative if it did an in-depth analysis of where the performance gain comes from. But Apple being Apple, I don't think they'll ever want to expose details on their internal systems, and we probably can only get such hand wavy statements.
I suspect that didn’t fit into the goal of the blog post.
I don’t think it’s meant to be a postmortem on figuring out what was going on and a solution, but more a mini white paper to point out Swift can be used on the server and has some nice benefits there.
So the exact problems with the Java implementation don’t matter past “it’s heavy and slow to start up, even though it does a good job”.
Sure, maybe you can get money to have some businesses try out rewriting their line-of-business software in the same language versus in a different language and get some results.
My expectation is that if you put the work in you can get actual hard numbers, which will promptly be ignored by every future person asking the same "question" with the same implied answer.
If the "just rewrite it and it'll be better" people were as right as they often seem to believe they are, a big mystery is JWZ's "Cascade of Attention-Deficit Teenagers" phenomenon. In this scenario the same software is rewritten, over, and over, and over, yet it doesn't get faster and doesn't even fix many serious bugs.
If the "just rewrite it and it'll be better" people were as right as they often seem to believe
Generally speaking, technological progress over thousands of years serves to validate this. Sure, in the short term we might see some slippage depending on talent/expertise, but with education and updated application of learnings, it's generally true.
I love to always see such comments. On JVM you use crap like Spring and over-engineer everything. 20 types, interfaces and objects to keep single string in memory.
JVM also like memory, but can be tailored to look okayish, still worse than opponents.
I'm hoping to hear some good news at WWDC for swift development in editors other than Xcode (VSCode, Neovim, etc.) Last year they said "we need to meet backend developers where they are" and announced plans to improve sourcekit-lsp and other efforts.
Xcode is probably like, one of the top… 3? 5? biggest macOS-native applications in the world.
Making it cross-platform would require either reimplementing it from scratch, or doing a Safari-on-Windows level of shenanigans of reimplementing AppKit on other platforms.
There would be no point in doing that. All the devs they care about already have Macs. If you are on Windows or Linux and you want to work on Swift server side code then you can use their official LSP or VSCode extension in your favorite editor.
Who even wants Xcode to be cross platform? People can barely tolerate it on macs.
What you really mean is people want the iOS development toolchain to be cross platform, and that would mean porting iOS to run in a hypervisor on linux/windows (to get the simulator to work). That is way too big a lift to make sense.
I’ve been writing iOS apps primarily for 15 years. I haven’t had to use Xcode since about 2016 since Facebook and Google have fully capable VSCode based editors with distributed builds in the Linux clouds. It’s pretty great, but I don’t know of an open source version of this system. That is, integration with Bazel/Buck that can build iOS on non-Mac hardware.
Swift SourceKit LSP + VSCode is actually pretty good these days. I recently got it working with CMake for a Swift / CMake project, and the only real pain in that was getting CMake set up properly. Everything else worked out of the box with VSCode's extension manager.
> One of the challenges faced by our Java service was its inability to quickly provision and decommission instances due to the overhead of the JVM. ... To efficiently manage this, we aim to scale down when demand is low and scale up as demand peaks in different regions.
but this seems to be a totally asynchronous service with extremely liberal latency requirements:
> On a regular interval, Password Monitoring checks a user’s passwords against a continuously updated and curated list of passwords that are known to have been exposed in a leak.
why not just run the checks at the backend's discretion?
> "why not just run the checks at the backend's discretion?"
Because the other side may not be listening when the compute is done, and you don't want to cache the result of the computation because of privacy.
The sequence of events is:
1. Phone fires off a request to the backend.
2. Phone waits for response from backend.
The gap between 1 and 2 cannot be long because the phone is burning battery the entire time while it's waiting, so there are limits to how long you can reasonably expect the device to wait before it hangs up.
In a less privacy-sensitive architecture you could:
1. Phone fires off request to the backend. Gets a token for response lookup later.
2. Phone checks for a response later with the token.
But that requires the backend to hold onto the response, which for privacy-sensitive applications you don't want!
Especially since the request contains the user's (hashed) passwords. You definitely don't want to be holding that on the server for longer than necessary.
Is it really a problem? Client can pass an encryption key with the request and then collect encrypted result later. As long as computation is done and result is encrypted, server can forget the key, so cache is no longer a privacy concern.
> why not just run the checks at the backend's discretion?
Presumably it's a combination of needing to do it while the computer is awake and online, and also the Passwords app probably refreshes the data on launch if it hasn't updated recently.
Without a deeper profiling analysis of the Java application it's hard to not consider this whole piece just advertorial content. Where exactly were the bottlenecks in Java or top delta gains compared to Swift. The service scope looks so simple that there might be some underlying problem with the previous version, be it the code not scaling well with the irregular batch nature of traffic or custom cryptography code not taking advantage of the latest native IO constructs.
And I'm not defending Java by any means, more often than not Java is like an 80s Volvo: incredibly reliable, but you'll spend more time figuring out its strange noises than actually driving it at full speed.
> Without a deeper profiling analysis of the Java application it's hard to not consider this whole piece just advertorial content.
I'd be surprised if anything Apple wrote would satisfy you. TFA makes it clear that they first optimized the Java version as much as it could be under Java's GC, that they evaluated several languages (not just Swift) once it became clear that a rewrite was necessary, that they "benchmarked performance throughout the process of development and deployment", and they shared before/after benchmarks.
Ok, I'm pretty skeptical that they actually did optimize what they could. In fact, reading between the lines it sounds like they barely tried at all.
For example, they mention G1GC as being better than what was originally there but not good enough. Yet the problems they mention, prolonged GC pauses, indicates that G1GC was not the right collector for them. Instead, they should have been using ZGC.
The call out of G1GC as being "new" is also pretty odd as it was added to the JVM in Java 9, released in 2016. Meaning, they are talking about a 9 year old collector as if it were brand new. Did they JUST update to java 11?
And if they are complaining about startup time, then why no mention of AppCDS usage? Or more extreme, CRaC? What about doing an AOT compile with Graal?
The only mention they have is the garbage collector, which is simply just one aspect of the JVM.
And, not for nothing, the JVM has made pretty humongous strides in startup time and GC performance throughout the versions. Theres pretty large performance wins going from 11->17 and 17->21.
I'm sorry, but this really reads as Apple marketing looking for a reason to tout swift as being super great.
I understand there may be some bias in this article, but the resource usage improvements are hard to ignore for a company that pays for cloud compute/memory usage.
I'm gonna look into server-side Swift.
Looks like it'll take some fiddling to find the right non-xcode tools approach for developing on linux.
I prefer Jetbrains tools over VSCode, if anyone has any hints in that direction.
I don’t know that there’s anything that magical about Swift in particular, but rather the difference of running without the big runtime of the JVM and the advantages of memory management. Go will probably give improvements (and likely be an easier mental adjustment from a JVM language), and Rust, once you mentally adapt to its model will give even bigger improvements. Swift has the advantage of being object-oriented (if that’s an advantage in your mind), but I was thinking 10 years ago that with the move to microservices that Java/Spring apps were going to not work so great with the quick start-up/shut-down that the cloud/microservice model wants.
Sadly, Jetbrains no longer sells AppCode or supports their Swift plugins for CLion. I WISH they would open source the plugins, as CLion is far superior to Xcode. For now, though, we're really stuck with either Xcode (Mac) or VSCode (wherever). That said, I am really starting to love server-side Swift using Vapor. Swift, as a language, is great to develop in.
I wonder what Apple is using for production monitoring, observability, and performance profiling with swift applications on Linux. In my experience this is one of the key missing pieces in Swift server ecosystem.
You can link against jemalloc, and use google perftools to get heap and CPU profiles, but it's challenging to make good use of them especially with swifts method mangling and aggressive inlining.
I've always wondered a slight amount as to why larger enterprises (which have the resources to hire specialists) don't hire people with expertise in say things like Rust, or Elixir/Phoenix?
It's one thing to say that we want to hire commonly available developers like in Java or C#, but if you have a long term plan and execution strategy, why not pick technology that may pay off larger dividends?
ITT: I get why they chose Swift, it's Apple's own in house technology. Makes total sense, not knocking that at all. Nice writeup.
It’s because Java and c# are so commoditised we don’t have to pay people as much.
Also it’s about short term balance sheet not long term product management in the SME and small SaaS space. I don’t think anyone other then the developers give a shit.
it sounds like you are asking why enterprises aren't choosing to write Rust code vs Java code. I wouldn't turn away a developer just because they had experience with Elixir.
It really comes down to whether there's someone making the case that a particular application/subsystem has specialized needs that would warrant hiring experts, and whether they can make the case successfully that the system should use technologies that would require additional training and impose additional project risk.
Until you are dealing not with enterprise applications but actual services, it can be difficult to even maintain development teams for maintenance - if your one ruby dev leaves, there may be nobody keeping the app running.
Even when you are producing services - if they are monolithic, you'll also be strongly encouraged to stick with a single technology stack.
No, I was making a point about if the goal is lowering resource usage, more throughput, concurrency, speed/perf, etc, a larger company can achieve that "unfair advantage" of things like rust or elixir etc because they have the resources as compared to SMBs or startups.
Of course, every company and org has to see whats best and feasible for them. Valid points you brought up no doubt.
Swift is Apple's own language. They have all the experts from lowest to highest level .
Writing a long winded report/article for fair technical evaluation of competing technologies would utter waste of time and no one would believe if answer were still Swift.
Apple maintains servicetalk[1] (java networking framework built on top of netty), so I'm guessing this is one potential JVM framework that was being used.
Readit News
As is always the case with such rewrites, the big question is whether the improvements came from the choice of language or because they updated a crusty legacy codebase and fixed bugs/bottlenecks.
Reduced memory consumption for cloud applications was apparently also the primary reason IBM was interested in Swift for a while. Most cloud applications apparently sit mostly idle most of the time, so the number of clients you can multiplex on single physical host is limited by memory consumption, not CPU throughput.
And Java, with the JIT and the GC, has horrible memory consumption.
Each extra GB per core you add to your shape, actually costs something. Hence every GB/core that can be saved results in actual cost savings. But even then, usually every extra GB/core is ~5% of the CPU cost. Hence, even when going from 10 GB/core (sort of a lot) to 1 GB/core, that only translates to ballpark ~50% less HW cost. Since they did not mention how many cores these instances have, it's hard to know what GB/core were used before and after, and hence whether there were any real cost savings in memory at all, and if so what the relative memory cost savings might have been compared to CPU cost.
You can share memory not only at the machine level, but between different applications.
Given that they also experienced a 90% reduction in Memory Usage (presumably from Java GC vs Swift AOT memory management) - it seems more likely the gains are in fact from the difference in languages.
If memory is an issue, you can set a limit and the JVM will probably still work fine
Create C like structs, in regards to memory layout segments, and access them via the Panama APIs.
I don’t think it’s meant to be a postmortem on figuring out what was going on and a solution, but more a mini white paper to point out Swift can be used on the server and has some nice benefits there.
So the exact problems with the Java implementation don’t matter past “it’s heavy and slow to start up, even though it does a good job”.
My expectation is that if you put the work in you can get actual hard numbers, which will promptly be ignored by every future person asking the same "question" with the same implied answer.
If the "just rewrite it and it'll be better" people were as right as they often seem to believe they are, a big mystery is JWZ's "Cascade of Attention-Deficit Teenagers" phenomenon. In this scenario the same software is rewritten, over, and over, and over, yet it doesn't get faster and doesn't even fix many serious bugs.
I confess to having been part of the cascade at various parts of my career.
It’s only Rust (or C++, but unsafe) that have mostly zero-cost abstractions.
JVM also like memory, but can be tailored to look okayish, still worse than opponents.
And https://github.com/swiftlang/sourcekit-lsp an be used in any LSP compatible editor like Neovim.
Making it cross-platform would require either reimplementing it from scratch, or doing a Safari-on-Windows level of shenanigans of reimplementing AppKit on other platforms.
What you really mean is people want the iOS development toolchain to be cross platform, and that would mean porting iOS to run in a hypervisor on linux/windows (to get the simulator to work). That is way too big a lift to make sense.
It was first released 7 years ago.
but this seems to be a totally asynchronous service with extremely liberal latency requirements:
> On a regular interval, Password Monitoring checks a user’s passwords against a continuously updated and curated list of passwords that are known to have been exposed in a leak.
why not just run the checks at the backend's discretion?
Because the other side may not be listening when the compute is done, and you don't want to cache the result of the computation because of privacy.
The sequence of events is:
1. Phone fires off a request to the backend. 2. Phone waits for response from backend.
The gap between 1 and 2 cannot be long because the phone is burning battery the entire time while it's waiting, so there are limits to how long you can reasonably expect the device to wait before it hangs up.
In a less privacy-sensitive architecture you could:
1. Phone fires off request to the backend. Gets a token for response lookup later. 2. Phone checks for a response later with the token.
But that requires the backend to hold onto the response, which for privacy-sensitive applications you don't want!
Presumably it's a combination of needing to do it while the computer is awake and online, and also the Passwords app probably refreshes the data on launch if it hasn't updated recently.
Deleted Comment
And I'm not defending Java by any means, more often than not Java is like an 80s Volvo: incredibly reliable, but you'll spend more time figuring out its strange noises than actually driving it at full speed.
I'd be surprised if anything Apple wrote would satisfy you. TFA makes it clear that they first optimized the Java version as much as it could be under Java's GC, that they evaluated several languages (not just Swift) once it became clear that a rewrite was necessary, that they "benchmarked performance throughout the process of development and deployment", and they shared before/after benchmarks.
For example, they mention G1GC as being better than what was originally there but not good enough. Yet the problems they mention, prolonged GC pauses, indicates that G1GC was not the right collector for them. Instead, they should have been using ZGC.
The call out of G1GC as being "new" is also pretty odd as it was added to the JVM in Java 9, released in 2016. Meaning, they are talking about a 9 year old collector as if it were brand new. Did they JUST update to java 11?
And if they are complaining about startup time, then why no mention of AppCDS usage? Or more extreme, CRaC? What about doing an AOT compile with Graal?
The only mention they have is the garbage collector, which is simply just one aspect of the JVM.
And, not for nothing, the JVM has made pretty humongous strides in startup time and GC performance throughout the versions. Theres pretty large performance wins going from 11->17 and 17->21.
I'm sorry, but this really reads as Apple marketing looking for a reason to tout swift as being super great.
No, it doesn’t.
I'm gonna look into server-side Swift.
Looks like it'll take some fiddling to find the right non-xcode tools approach for developing on linux.
I prefer Jetbrains tools over VSCode, if anyone has any hints in that direction.
You can link against jemalloc, and use google perftools to get heap and CPU profiles, but it's challenging to make good use of them especially with swifts method mangling and aggressive inlining.
It's one thing to say that we want to hire commonly available developers like in Java or C#, but if you have a long term plan and execution strategy, why not pick technology that may pay off larger dividends?
ITT: I get why they chose Swift, it's Apple's own in house technology. Makes total sense, not knocking that at all. Nice writeup.
Also it’s about short term balance sheet not long term product management in the SME and small SaaS space. I don’t think anyone other then the developers give a shit.
There are not enough Rust experts in the world for a typical enterprise to hire and benefit from it.
Elixir/Phoenix are not order of magnitude improvements over Java like Rust, they are marginal improvements. Enterprise don't care for that.
It really comes down to whether there's someone making the case that a particular application/subsystem has specialized needs that would warrant hiring experts, and whether they can make the case successfully that the system should use technologies that would require additional training and impose additional project risk.
Until you are dealing not with enterprise applications but actual services, it can be difficult to even maintain development teams for maintenance - if your one ruby dev leaves, there may be nobody keeping the app running.
Even when you are producing services - if they are monolithic, you'll also be strongly encouraged to stick with a single technology stack.
Of course, every company and org has to see whats best and feasible for them. Valid points you brought up no doubt.
Was the Java Service in Spring (boot)?
What other technologies were considerd?
I'd assume Go was among them. Was it just the fact that Go's type system is to simplistic or what were the other factors?
Writing a long winded report/article for fair technical evaluation of competing technologies would utter waste of time and no one would believe if answer were still Swift.
> I'd assume Go was among them. ...
I don't see any reason to evaluate Go at all.
https://devblogs.microsoft.com/typescript/typescript-native-...
[1] https://github.com/apple/servicetalk