Checkout the async gem, including async-http, async-websockets, and the Falcon web server.

https://github.com/socketry/falcon

Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default adapter to use async_http with falcon and async-job instead of thread-based approaches like puma and SolidQueue. This should significantly improve resource efficiency for AI workloads in Ruby - something I'm not aware is implemented by other major Ruby LLM libraries. The current approach with blocks is idiomatic Ruby, but the upcoming async support will make the library even better for production use cases. Stay tuned!

    # Stream responses in real-time
    chat.ask "Tell me a story about a Ruby programmer" do |chunk|
      print chunk.content
    end

Langchain really, I think, pushed the LLM makers forward toward adding those features but unfortunately it got left in the dust and became somewhat of a zombie. Simultaneously, the foundational LLM providers kept adding things to turn them more into a walled garden, where you no longer needed to connect multiple things (like scraping websites with one tool, feeding that into the LLM, then storing in a vector datastore - now that's all built in).

I think Langchain has tried to pivot (more than once perhaps) but had they not taken investor $$ early on (and good for them) I suspect that it would have just dried up and the core team would have gone on to work at OpenAI, Anthropic, etc.

It doesn't deal with any of the hard problems you'll routine face with implementation.

I'm pretty sure this API semantics (instance builder to configure, and then it's ask/paint/embed with language-native way to handle streaming and declarative tools) would look beautiful and easy to use in many other languages, e.g. I can imagine a similar API - save, of course, for the Rails stuff - in Python, C# or Erlang. While this level of API may be not perfectly sufficient for all possible LLM use cases, it should certainly speed up development time when this level of API is all that's possible needed.

    // Just ask questions
    const chat: Chat = LLM.chat;
    chat.ask("What's the best way to learn TypeScript?");
    
    // Analyze images
    chat.ask("What's in this image?", { image: "ts_conf.jpg" });
    
    // Generate images
    LLM.paint("a sunset over mountains in watercolor style");
    
    // Create vector embeddings
    LLM.embed("TypeScript is powerful and scalable");
    
    // Let AI use your code
    class Calculator {
      description = "Performs calculations";
      params = {
        expression: { type: "string", desc: "Math expression to evaluate" },
      };
    
      execute(args: { expression: string }): string {
        return eval(args.expression).toString();
      }
    }
    
    chat.withTool(new Calculator()).ask("What's 123 * 456?");