Readit Newsreconditerose commented on Redis is open source again antirez.com/news/151... · Posted by u/antirez
I wonder whether they did deals with AWS and Google behinds the scenes. Something like "you poured money into Valkey, how about you give that kind of money to us instead and we'll switch to AGPL and you can stop confusing two customers with two practically identical but differently priced options". Could that ever work? (I have no idea)
Google is well known for having banned AGPL (https://opensource.google/documentation/reference/thirdparty...).
Garnet fascinates me. Their benchmarks even claim that it is better than Redis and also Dragonfly. Are there any papers or write ups explaining what makes Garnet fast? (I do know its based on FASTER)
The tl;dr is it's just a lockless hashmap attached to a TCP server with a log. Simple Get/Set operations are highly optimized, so with high batching they are able to efficiently fetch a lot of data efficiently. The architectures scales very well when you add threads and data access that is uniform.
It struggles a bit on certain types of workloads like hot keys, think heavy hitting a single sorted set. It's a cool architecture.
1. The fundamental idea was mine. Ideas can be improved, of course, and that's good. That was my point. However, allow me to say that Redis is fundamentally a "software of ideas" than anything else. Technologically it is far from impressive (Redis, ValKey, all the forks).
2. Redis 8 improves the same idea, too, released today.
3. If you claim [in a different comment here] you provided a lot of code to Redis, why you didn't send a pull request for that? So, you are practically saying you were using, at Amazon, all the BSD code we provided, but could not provide an important part of the code to us? You see how broken such model was? At least stop defending it.
4. We can now copy the implementation: the parts are reversed (the irony!), and your code is BSD as our was for 15 years. When we avoid doing things like that, is because we have issues with how certain things were made.
5. I don't understand the motivations of you and other AWS people commenting here today. You work for a company that is creating issues to the OSS ecosystem: this is hard to deny. You cloned (and, yes, the license allowed for it) the code of Redis, and work on it so that hyperscalers can continued to do what they used to do. We bring Redis back to AGPL, and you are here to do the interests of Amazon in the comments. Did you see me commenting your stuff, when you release your things, with comments like "ah! But this is unfair"?
There is to make choices. I understand that it was cool to continue to work at a Redis fork, and part of the incredible thing open source is, is that forks survive in the hands of different teams (but design ideas can be misunderstood and projects may turn into other projects). So if you are happy to hack on ValKey, I hope you'll have the best experience out of it. But there is to make choices on how/when to interact.
> So, you are practically saying you were using, at Amazon, all the BSD code we provided, provide an important part of the code to us? You see how broken such model was? At least stop defending it.
I'm not defending it, I'm trying to fix it. I want Amazon to contribute back. That's what I spend most of my time doing, but I can't just sit in a meeting and tell people we should give away code. It takes time to convince people that we should collaborate on the core and just compete on what we want to differentiate on. It takes time to convince people that building open-source in a vendor neutral space makes software that is better for everyone.
I hope that makes sense.
Are there usage stats available? How do you know this?
My guess is they are making it up. AWS has no public information, but there are some high profile customers that have migrated https://aws.amazon.com/elasticache/customers/.
AWS and GCP offer valkey-based versions of products that are typically based on Redis, but those versions are currently, generally, preview-grade, and statistically zero customers are using them. They still offer the original, Redis-based versions of those products, which, statistically, 100% of their customers are using.
Do you have data to back up your claims? I see a lot of customer claims for Valkey here, https://aws.amazon.com/elasticache/customers/. Neither of the AWS or GCP offerings are in preview.
The I/O threading was initially implemented by myself, when I was still working on the core before rejoining, and is also available in Redis. So this is actually another case of the benefits ValKey got for being able to clone Redis, receiving money from the hyper-scalers that just want to maximize their revenues...
They made certain improvements later, but Redis 8 (see the release notes in my blog post if you are curious) improved this stuff a lot, too.
Also, Redis 8 contains a new data structure, Vector Sets, that allows to do many useful things, together with more probabilistic data structures, single hash fields expires, and many other stuff. It is not factually correct that ValKey has any features edge over Redis I believe. We will see in the next year which project takes the best development path: this is what really matters.
The Valkey implementation of multi-threading is fundamentally different than what existed in Redis. The history dates back to work done in ElastiCache that was released as "Enhanced IO", https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-el.... The version released in Redis could only do about 350k RPS because of poor memory locality of operations, the inability to do command processing while handling I/O, and the inability to offload much of the TCP read path. The new version in Valkey can achieve 1.2M RPS.
"They made certain improvements later", should be "we threw away the old implementation and built a better one."
It is a fork of the latest oss release right? I thought some completely new implementations were introduced.
It's not just a fork, there have been two releases on Valkey that improved performance and memory efficiency. There is a lie that Redis likes to spread that only their own employees were working on the core engine at the time of the fork, but most of the engineers on Valkey came directly from having worked on Redis OSS. A recent example is we modernized the hash table a bit: https://valkey.io/blog/new-hash-table/.
I am thinking the same that going to AGPL may actually push more people to Valkey.
Although I haven't checked if ValKey any substantial development since the fork.
Yeah, there has been a lot of stuff like performance [1] and efficiency improvements [2]. A lot of the contributors, that didn't work for Redis labs but worked on Redis OSS before the fork, moved to Valkey and they continued to contribute.
[1] https://valkey.io/blog/unlock-one-million-rps-part2/ [2] https://valkey.io/blog/new-hash-table/