Rama is a testament to the power of Clojure

metadat · 2 years ago

> Rama can build end-to-end backends at any scale on its own in a tiny fraction of the code. At its core is a new programming language implementing a new programming paradigm..

Grand claims, but where are the super powerful demo apps? The getting started docs are a trickle of info..

https://redplanetlabs.com/docs/~/operating-rama.html#_access...

The "demo gallery" has a few cherry picked examples, but I don't see how this fulfills the claim of a "new paradigm". I wanted to be blown.. but this ain't it.

https://github.com/redplanetlabs/rama-demo-gallery

Oof.

nathanmarz · 2 years ago

Check out twitter-scale-mastodon, which is an implementation of Mastodon's backend from scratch that scales to Twitter scale. It's more than 40% less code than Mastodon's backend and 100x less code than Twitter wrote to build the equivalent.

https://github.com/redplanetlabs/twitter-scale-mastodon

https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...

finnh · 2 years ago

> These numbers are a bit better than Twitter’s numbers. Because of how unbalanced the social graph is, getting performance this good and this reliable is not easy. For example, when someone with 20M followers posts a status, that creates a huge burst of load which could delay other statuses from fanning out. How we handled this is described more below.

Huh. I haven't written myself a twitter, but I've always assumed that this problem is solved by a mix of fan-on-write and fan-on-read. Most people are fan-on-write, but those with a follower count > $THRESHOLD are fan-on-read.

The fan-on-read set is small enough to be easily cacheable, so the extra reads required to service a follower's feed are more than compensated for by the fact that the content is in RAM in a cache.

Sounds like RedPanda went exclusively with fan-on-write. I'd be surprised if Twitter does the same - but it sounds like OP has more knowledge about this than me, so I'm happy to be educated!

metadat · 2 years ago

> Running on a real cluster

> Until Rama is publicly available, the only way to get access to the full Rama release is by applying to our private beta.

I thought it was coming out last August? What happened?

(As per your link, https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...)

Edit: @nathanmarz that's cool you have enough interest to keep things moving with it being private. I'm only interested in open solutions for this domain, though. IMO, accepting lock-in is a lot to ask. Best wishes ~ MD

ntonozzi · 2 years ago

Here's an example of the backend code:

https://github.com/redplanetlabs/twitter-scale-mastodon/blob...

It certainly looks like it does a lot with their DSL, but as a newcomer it's really hard to parse.

nathanmarz · 2 years ago

I suggest starting with the tutorial and the heavily commented examples in rama-demo-gallery, linked below.

https://redplanetlabs.com/docs/~/tutorial1.html

https://github.com/redplanetlabs/rama-demo-gallery

geodel · 2 years ago

Well there is whole bank transfer implementation there and thoroughly commented too. Some entrepreneur can take this and run with it to create next generation banking platform.

sokoloff · 2 years ago

Maybe they could call their new Clojure-based bank Nubank.

kitd · 2 years ago

Tbf, the purpose of the blog is to talk about the power of Clojure/Lisp for defining the new paradigm, rather than the paradigm itself. Though the author then goes on to talk about "fragments" as the basis for the new paradigm.

The first sentence has a link to the project page which has most of the info you need.

smackeyacky · 2 years ago

Not even a code sample in the article. You'd think if they were so convinced it was a new paradigm we'd see why. Makes me skeptical.

nathanmarz · 2 years ago

It's an article about Clojure's influence on Rama, not an introductory tutorial on Rama. But we have plenty of code samples elsewhere:

https://github.com/redplanetlabs/rama-demo-gallery

https://github.com/redplanetlabs/twitter-scale-mastodon

https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-...

https://redplanetlabs.com/docs/~/tutorial1.html

cactusfrog · 2 years ago

The Clojure community is extremely cult like. I worked with a dev who tried to push Datomic, and despite the huge flaws with his system, and the fact that it was closed source, he would continuously tout how much better it was than a relational database. I don’t believe the hype.

lispisok · 2 years ago

You just generalized an entire community based on one person. I can add a counter anecdote to your N=1. I'm a Clojure dev. I've used and enjoyed Datomic. Postgres is still my goto database.

undershirt · 2 years ago

I know our engineering discipline is in shambles, but there are probably better ways to argue against hype without ourselves using hyperbole.

jabradoodle · 2 years ago

N=1, I work with many pragmatic clojure devs who just want to ship working software.

lgrapenthin · 2 years ago

There are no huge flaws with Datomic.

jimberlage · 2 years ago

Is anyone using Rama at scale? This isn't a complaint about it being closed-source, people have to make their money somehow. But I've seen a lot about Rama's scale when the only things to check the claim are a private beta and a single-process dev build.

I'm super curious to hear from people who have tried it for larger projects!

nathanmarz · 2 years ago

A few of our private beta users are using Rama at larger scale (>1M appends/second). We'll be publishing case studies eventually.

joeevans1000 · 2 years ago

This is... closed source?

Omg. Didn't anyone learn anything from the Datomic story?

thih9 · 2 years ago

What do you mean?

mrkeen · 2 years ago

> Another example is Specter. Specter is a generically useful library for querying and manipulating data structures

> https://github.com/redplanetlabs/specter/tree/master/src

> MutableCell.Java. 8 years ago

  package com.rpl.specter;

  public class MutableCell {
    private Object o;

    public MutableCell(Object o) {
      this.o = o;
    }

    public Object get() {
      return o;
    }

    public void set(Object o) {
      this.o = o;
    }
  }

Behold the power!

Deleted Comment

thih9 · 2 years ago

Simple and elegant, powerful indeed.

Or did you mean something else? In which case please elaborate.

titanomachy · 2 years ago

> At its core is a new programming language implementing a new programming paradigm, at the same level as the “object-oriented”, “imperative”, “logic”, and “functional” paradigms... Rama generalizes the concept of a function into something called a “fragment”... a fragment can output many times (called “emitting”), can output to multiple “output streams”, and can do more work between or after emitting.

This sounds really similar to programming with observables (e.g. RxJS), is a "fragment" an even more general concept?

I used observables when I did frontend development, I found it a useful paradigm (once I got my head around the idea).

nathanmarz · 2 years ago

A fragment itself is a generic programming construct that serves the same purpose as a function (just more general). When used in Rama topologies, they serve a similar role as observables in terms of reacting to new data as it flows through and sending any amount of information downstream to any number of output streams.

ThinkBeat · 2 years ago

I remember when the first time i read about Rama, with the blog post about having duplicated Twitter scale at 100x code or some such.

Which all in all, if you have built a platform/runtime/language and you pick one particular example to highlight it, being able to achieve XXXX% less than is trivial.

I was curious about all the bluster then, Seeing this post I am somewhat disappointed that there does not appear to have been a lot of visible progression.

But I guess it is still running semi stealth in private beta.

I will be interesting to see what becomes of it.

nathanmarz · 2 years ago

Indeed, we're in private beta and aren't publicizing much about what we're doing. We'll eventually be releasing many case studies on how our private beta users are using Rama.

MH15 · 2 years ago

I think we've spent a lot of time looking at backend and frontend web app development as separate. In my work, TRPC (https://trpc.io/) solved this. I hope others in different frameworks find a similar solution soon.

JonChesterfield · 2 years ago

Fragment sounds like actor to me. Stateful thing that sends messages to other fragments. Not convinced that's a generalisation of a function.

nathanmarz · 2 years ago

Fragments have nothing to do with actors. Check out the doc on the dataflow language and you'll clearly see it's a generalization of a function.

https://redplanetlabs.com/docs/~/clj-dataflow-lang.html

samsquire · 2 years ago

From just thinking about the architecture of Rama I feel it would scale very well.

Data parallelism and partitioning and sharding is a very effective scaling technique.

Nathan, I would appreciate writings about the mental model of mapping software to your mental model of implementing behaviours in the streaming data approach because it is a different paradigm.

I can't read the DSL yet and know what is going on!

nathanmarz · 2 years ago

The best documentation on the mental model of using Rama is the last page of the tutorial, linked below. However, I would recommend going through the whole tutorial rather than starting there.

https://redplanetlabs.com/docs/~/tutorial6.html