In my last role as an engineering manager for a B2B-oriented product, I authored and reviewed many diagrams for backend applications, mostly for integrations between 2 third-party services. Some of these diagrams were elaborate enough that I started dreaming of a way to simply run a diagram as is; I imagined a “run” button on the top-right corner of the screen that would execute the diagram without the need to translate it into code.
That led me down a rabbit hole of exploration and experimentation, from tools like Zapier, Pipedream and Make, which are great for automating “backoffice” stuff, and up to NodeRED, NoFlo.js and the great work of J. Paul Morisson on Flow-Based Programming. I failed to find a tool that would answer my needs - a tool that balances a new level of abstraction, manages to stay powerful and flexible, and most importantly, integrates with the existing ecosystem, and doesn’t replace it. I built Flyde as an attempt to answer that need.
Flyde is designed to complement and enhance traditional textual coding, not to replace it. It includes a VSCode extension, it seamlessly integrates with existing TypeScript/JavaScript code and can run on Node.js and in the browser.
I believe that as we delegate more coding tasks to AI, we’ll assume the role of an architect rather than a programmer. This shift will require tools that focus more on orchestration and high-level troubleshooting and less on low-level functionality.
I’d love to hear your thoughts and feedback on Flyde’s direction!
I know it's polarizing but I truly think that visual programming remains an entirely unexplored area. I'm convinced that the current state of "text" programming is totally ineffective and in fact we are the last domain producing things with computers which still insist in being limited by the text files in folders abstraction.
It's a shame that in 2024, I still have to search for text in files like it's 1970, guess which file does what based on the dozen of characters of the file name and can't see at a glance which other files are dependencies or uses.
I can't "see" my entire codebase, zoom in and out, I still have to guess the relation between some line of code and another in another file.
My ideal IDE of the future just allows me to see all my codebase like a big fractale.
Visual programming languages have been around since at least the 1970s:
https://en.wikipedia.org/wiki/Visual_programming_language
Several are used real-world production cases:
https://en.wikipedia.org/wiki/LabVIEW
https://en.wikipedia.org/wiki/Simulink
https://en.wikipedia.org/wiki/Pipeline_Pilot
https://en.wikipedia.org/wiki/IBM_Cognos_Analytics
While visual programming systems work better for some use cases, they are usually less agile versus text-based systems for several generalized programming tasks: 1) difference comparisons ("diffs"), 2) version control, 3) code search, and 4) code input (Most visual systems require a mouse and careful placement of connectors).
Visual programming systems tend to excel for domain-specific tasks carried out by non-technical or semi-technical users, but for generalized programming, text-based programming systems are more popular for highly technical software developers.
In my experience real code can't be represented legibly on a 2D plane. "Generate code map" features of IDEs usually produce incomprehensible graphs, when you try them on actual codebases.
I never tried CodeSee before it shut down, was it any better?
https://successfulsoftware.net/2024/01/16/visual-vs-text-bas...
It agrees with quite a few of the points you make above.
They're used everywhere - anywhere you see a shiny metal cabinet with conduit running in and out, there's a chance there's a PLC plugging away in there.
Ladder dates back to the 70s and I'm willing to bet is the most used graphical language in existence. It looks like the relay diagrams that electricians use.
I don't have to care how you're getting a random point in a sphere. Whatever algorithm you want to use works for me. And since you know which algorithms work better for which scenarios, you can put all of that delegation under the hood. All I have to know about is which node to use. And if something starts breaking, I know that it can't be my problem; it has to be a problem with the node's code and that domain professional can address it.
Of course, a good dev might say "That's just having a really good API", which is true! But once you have an API so good that you don't need to know any of its internals, you're essentially trying to write a graph using documents, which is kind of silly. High-level servers are a great example (node servers, python servers, etc). There's not any utility in writing an expressJs server with text that isn't satisfied by writing it as a flow chart. Servers, at that level of abstraction, are so simple that all you really need to do is tell which keys go where, when. And that's most simply done by drawing boxes and arrows that all point to each other.
Put more starkly: there's no difference between writing `Lib.Physics.GetPointInSphere(param1, param2, param3)` and linking a `GetPointInSphere` node to `param`s 1, 2, and 3. So I think that if you're in a domain that is already at that level of abstraction, visual programming is a fantastic way to go. And if you're not already at that level of abstraction, there's value in getting there (even though it's probably going to require fracturing/modularizing existing concepts).
And I think flow diagrams are a terrible way of describing many algorithms.
Flow diagrams look great for multi-processing pipelines and event suff.
I'm interested in this as an addition to current coding practices.
- Simplicity: every computer and most humans can interptet it without any issues
- Diffing: it's relatively easy to tell what changed in a text file between revisions
- Editing tools: text editors, formatters, etc. exist and many people know how to use them efficiently
Also, modern developer setups allow you to manipulate code as a syntax tree (see tree-sitter text-objects) and intellisense and snipping tools allow you to type much less.
If we spent the time building tools for visual coding it’s possible we’d be able to do way more.
Just thinking about diffing for example, it would be much easier to see which nodes have changes in the whole codebase by just highlighting them red. It’d be easier also to depict something like “the flow for this process changes from this to this”
Simulink has been around and in heavy use for at least 2 decades but I think the $20k/yr price point is why we don’t see many other applications of it (even though people HAVE written games and web apps and such with it)
[1] Rethinking Visual Programming with Go - https://divan.dev/posts/visual_programming_go/
PLEASE PLEASE PLEASE PLEASE PLEASE make the nodes snap to a larger grid, and have the grid browsable with the keyboard.
I'd rather use Factorio as an interface than an interface where the nodes and edges just float around.
Excel is a DAG that is browsable with the keyboard (Ctrl+[ anyone?), and that's proven very usable.
- As a general design, I prefer the integrated "function with parameters" node type found in systems like Unreal's "Blueprints" far better than having to manage the parameters and functions separately from each other. It's more cumbersome to develop, but no less flexible to use, and loads more simplified. A distinction for the code flow from the data referencing is helpful for me, at least.
- It doesn't fit my exact needs, because I need something that allows me to expose this functionality to users. Essentially, I need something that does what your playground does, as a library. The interoperability is perfect, though! Allowing users to set up flows and then being able to simply import them rather than having to translate them is a fantastic DX. I guess, at the end of the day, I would still like clean, nicely formatted Javascript (not typescript), so I imagine your library isn't really suited for that, either. But I definitely like the architecture of it!
Luna Park was indeed a npm package that you could integrate to allow users to build their own logic using visual scripting. In the end, I pivoted, and Luna Park is now a visual scripting wrapper around the Vue.js framework, allowing people to build modern webapps without code. (I also made https://roller-coaster.app to create endpoints using the visual scripting system of Luna Park)
In any case, that's super cool to see people building awesome tools like yours in the visual programming space :) ! I love how Flyde show you what node is running, and the way it executes logic is really interesting!
More than that, though, it's about the difference between 'data references' and 'execution flow'. It's something that Unreal makes very, VERY obvious. Color coding data types, emphasizing execution flow lines, etc. I can't recommend using it as a guide, enough. Whatever problem I've come up with as 'complicated' for visual scripting, they've got an elegant solution for.
Even stuff like being able to 'break' nodes into other nodes, or pins into other pins. So a "Point" node might be useful for sending into a function expecting a Point data type, but you might also want to send just that Point's x or y value to something else. In that scenario, you might 'break' the exit pins into a Point reference, an x reference, and a y reference. It would be cumbersome and unruly to always have that available. But the UX to make it available is straightforward and satisfying.
And then, all of that aside, I just prefer the visual look of pins integrated into the nodes. The extra lines you get from drawing between the pin and the node is messy and since you can't move them independently, they don't really add anything. Just a lot of extra buffer to prevent a 'crowded' feeling. But at least that's not something unique to Unreal. I don't think I've ever seen a visual scripting system that used pins as separate elements. So maybe it's just saturation bias. But I do like what I like!
Anyway, I'll definitely mess around with it and see if it can help with its embedded version. To be completely honest, I think there's too much friction, overall. I would need something to render as plain javascript, rather than a JSON structure or minified JS (or typescript), so there's a translation there. And then to get it graphically how I want it would be complicated due to all the domain-knowledge required for integrating with your app's underlying library (looks like next js? Not sure, but it's not something I want to work with). Not to be too pessimistic or anything. Most libraries work toward providing a complete solution, not towards providing a composable, compsitable, modular set of utilities, which is what I need. I don't hold any contempt for providing what you've provided because it's a hell of a thing and perfect for a lot of use cases, I'm sure! I'm just very pragmatic about my own use-cases, and my development quirks (like not using libraries that obfuscate functionality [next, react, svelte, etc]).
My attempt on this is https://openexus.com where the goal is really to create some form of universal plug-and-play building blocks. Your approach is almost very low-level with direct translation to code. My attempt is slightly higher-level (but developers can create as low level as they want). More importantly, the visual diagram build on openexus is a reactive graph (almost like spreadsheet), not a sequential directional flow graph (like node-red, or yahoo pipes).
Would love to chat if you are up for it. m at lominming dot com.
I started closer to this at first (but still lower-level) and slowly got "down to earth", taking a https://www.dreamsongs.com/WorseIsBetter.html approach.
I'm curious to know more of what you found lacking in the various visual programming languages? You mention a few general things but (other than the integration angle) I'm having a hard time understanding exactly what limits you hit with the other options that caused you to build your own.
And to add some context to the above questions, is this primarily your own research or do you anticipate it being used for production systems?
NoFlo.js was the closest to allowing this, but I think that it was too early for the game, and NoFlo took a non-integrative approach. Vladimir Sibirov wrote vastly about that and why he thinks it failed in this great blog post - https://blog.kodigy.com/post/state-of-flow-based-programming... I was happy to see Flyde addressed this
Other purely FBP implementations did not put enough emphasis on the visual aspect, which for me was crucial to nail on a holistic level.
And for your last question - my goal is for it to be used in production systems, yes. I plan to release a Flyde-based visual API builder soon - https://www.trigg.dev as a more specific use-case, and hope that the fact you can download the Flyde flows and run them wherever you want will help potential users overcome the fear of using a low-code tool.
Is your project really better than NodeRED? I mean it does have over 4000 community contributed nodes.
25 years ago when I read that Microsoft has a software development environment named "Visual Studio" I imagined, well, something with graphs and nodes and flow and... Well something visual, right? Now it's 2024 and VS is still not visual.
Of course things are not as simple. Flyde (along with many alternatives mentioned) is visual IMHO. Keep up the good work!
I do love visual programming and I use n8n a lot for my side projects. I really like its "delayed debug" features, so that I can analyse each step of the flow weeks later than it happened (i.e, I can see why a webhook failed long ago and even replay it step by step).
One missing feature that I've been working on is a "export workflow to code" feature. This way, once you are finished working on a workflow, you could run it everywhere without the need of installing the full IDE.
Again, nice work!
``` import { loadFlow } from "@flyde/runtime";
const execute = await loadFlow("./celsius-to-fahrenheit.flyde");
const inputs = { celsius: 0 }; // "celcius" is a main input in the flow, therefore it must be provided when executing the flow const { result } = execute(inputs); // execute returns a "result" promise, along with a cleanup function that can be used to cancel the execution.
const { fahrenheit } = await result; // each output in the flow is a property on the result object
console.log(fahrenheit) ```
(taken from https://www.flyde.dev/docs/integrate-flows/)
But your comment strengthens my feeling that making this more intuitive and discoverable and is indeed something I should prioritize
1. A new language. New languages don't go anywhere 99.9% of the time and when they do it takes a massive undertaking by hundreds of people as well as very careful design over the course of decades.
2. Building visually at the expression level. Writing expressions in any modern language is very compact. In this case what takes up a single line now takes up and entire screen.
if(n>1) return fib(n-1) + fib(n-2);
Graphs are much better for working at a high level because that's where the locality isn't there are it isn't clear what data is going where. For expressions drawing lines instead of just using the same variable that was used in the previous line doesn't help clarity.
This gets in to another problem I didn't mention - branches and loops (not to mention class declarations).
Feeding a function result into another function argument is great, but that was never difficult for expressions anyway. Once you get into branches and loops you have another problem that is a huge problem for nodes.
Ultimately programs don't map to data flow nodes, they are imperative. Certain parts of certain programs will end up mapping very well, but when looking for silver bullets in something that isn't 1:1, you end up hacking it up and forcing a square peg in a round hole to make the dream work.
GraphSCAD addressed that by allowing functions to have custom icons.
I'd really like to see that become more prevalent.