I believe already some 2 decades ago Bill Gates said that the biggest failure in software is the lack of productivity gains in software development.
Just to cherry pick a dramatized example. In the late 90s, one of my first programming experiences was in Delphi. It's a very visual way to program. You drag and drop a UI together from standardized elements. You could data bind things like input fields to a data source. You'd double click a button and it takes you to the already created "onclick" event where you do the actual coding. In a way, this is still similar to how you can "program" Microsoft Access today.
In this same period (I was in a computer science study) I was moaning to my teacher how tricky programming in C was. I made tiny memory management errors crashing the entire PC, with zero debug output.
His words: "Don't worry. By the time you get to work, you'll just do modeling".
Now I work on the web. It's been 30 years since Delphi and instead of productivity gains, we've taken steps in the opposite direction. There is no standardized UI library. JavaScript has no standard library. There's no single unified architecture. There's no standard tool chain. It's all very low level after which we stitch together fragile semi-abstractions.
I know it's an imperfect comparison, but I hope it gets the point across. It's all far too low level and fragile.
This doesn't just hurt productivity, also accessibility. It's simply not beginner friendly.
At least from the systems programming perspective I think productivity gains have been significant. Valgrind had _just_ appeared as I was doing my undergrad and it allowed me to focus on the high-level math of data structures -- as an example -- compared to my non-Valgrind using classmates that were struggling with memory bugs (which, of course, I did too, but to a less degree). C++ has noticeably improved since then, if you can hew to the newer bits of the language and keep all the arcane bits in your mind. Rust has arrived on the scene, inspired by Cyclone, C++, SML and is a serious jump in productivity over C++ because the compiler keeps track of all those arcane bits. If you squint, Go's in the mix for web stuff that doesn't mind GC. It's a shame ATS didn't become more of a thing, Ada has made real strides back from the dead, especially if you can stick to SPARK Ada.
We're even seeing more formal verification of complex bits of software in systems-land, something that seemed basically impossible when I first read about Coq twenty something years ago.
I do admit, though, that front-end seems like a wild west presently, to my ignorant eyes.
> Ada has made real strides back from the dead, especially if you can stick to SPARK Ada. We're even seeing more formal verification of complex bits of software in systems-land, something that seemed basically impossible when I first read about Coq twenty something years ago.
I really agree with this. The situation is a bit crazy. Since formal proofs are hard, the alternative taken is giving no guarantees at all. That's bad. Static analyzers can get really far. Contracts, even just at runtime are really useful. I don't want a piece of code to run if the precondition is not met, and I don't want it to return or have side effects if the postcondition is not met. This would rule out a massive class of errors. Of course, bonus points if you can do it at compile time.
Ada SPARK, Infer [1] and Dafny [2] show that rigorous engineering is possible and practical. Actually, Facebook (Meta now) argues Infer let's them move faster.
I think the problem is political and economical. It's hard to monetize this first. I wish someone like Apple took the plunge and rebuilt a simpler version of their stack using formal verification. It's the kind of thing only a large corporation or a government can do because it might take half a decade to take off. Once it takes off its unstoppable. Who would not like to buy e.g. a phone where most bugs and security breaches are impossible by construction? Verification also has a nice side effect. Change for the sake of change and featuritis are pretty incompatible with it.
"It's a very visual way to program. You drag and drop a UI together from standardized elements. You could data bind things like input fields to a data source. You'd double click a button and it takes you to the already created "onclick" event where you do the actual coding. In a way, this is still similar to how you can "program" Microsoft Access today."
Yes! I remember Delphi as my first language in school, too. And later in university I could not understand, why in all the other advanced languages like Java, C and C++ doing the same was so hard.
Then I discovered Flash and later Flex with Flex Builder. That was a even way more powerful experience.
You could draw up anything - and make a button out of it with a click. Or a sophisticated, reusable component.
And then it died, because of Adobes unwillingness to open source the technology (and their focus on performance and flashy features and not security). All the bad flash applets that were flying around existed, not because it was so bad, but because it was so easy even for total beginners to make something working fast.
Regarding Java and C#, I am stunned by how good are IDEs now. To be clear, when I say IDE, I am specifically talking about a tool that enhances the edit-compile-debug cycle. Compared to 20 years ago when I wrote C, C++, and Perl, the IDEs were mostly terrible, except Visual Studio for Windows. The majority of my development cycle was all terminal-based: vim to edit, make to build (C/C++ only), gdb/perl -d to debug. Hoi, awful. (And, I still enjoy those all three languages!) As mega-corp computing slowly migrated to Java & C#, my productivity has increased dramatically. I guess at least 2x, if not more. Yes, some of this comes from how forgiving a modern VM can be (NullPointerExceptions, debugging, etc.), but a lot can ascribed to the incredible ecosystem of IDEs.
Regarding JavaScript, in the context of a modern web browser, high school students (in almost any country in the world) can easily Google a little bit, read some blogs, and then start GUI programming. And this doesn't require expensive, licensed software, like Delphi 30 years ago.
The web, and much of programming really, feels so far behind the curve of what's possible as a programmer. Take a dive into shader design in something like unreal engine 5, it sounds exactly like what you're describing in Delphi, mind you, I say this as someone who has never programmed in that language. The ability to drag and drop functions, compose higher levels of abstractions, and really all of the fun programming stuff we do can be accomplished in these engines. I've found it very enlightening, and even somewhat frustrating, to go from drag/dropping some graph nodes that generate a playable world for fun, and then back into the world of javascript for work where I am slinging 100s of lines of text-based code just to get a form button to operate correctly.
In game engines, and especially in UE where things like Blueprints exist, the low level stuff is just conveniently hidden or abstracted away. But the long textual code that makes the button work, eventually exists somewhere in there. At least when you write it yourself, you know who to blame (and probably where to look) when it doesn't work.
I've used UE4 for years and the Blueprints system doesn't save you much. Those abstractions are bricks, made for building brick houses. If you want to build something made out of anything other than brick, you're back to custom HLSL nodes and better off programming your shader somewhere else.
Blueprints are nicer for known problems, they are a better glue for old knowledge, but when you want to do something custom or new, you're out of luck.
This is the same problem with all programming, you can't make something specific and new without doing an engine teardown and rebuilding it. All the parts are terminally interconnected and always will be.
At the end of the day, UE4/5 is a landing pad for all the work you make in other tools. I agree web dev is no fun though.
Ironically, I know everything from ASM to...well many languages, including C, C++, PHP, Python, Ruby, Javascript, etc.
The most productive tools I used prior to learning all of that? HyperCard on the Mac, Visual Basic on Windows. Why? Tooling. Even when you had to get down and dirty, it seemed the tooling helped you out in some way. Shoot, I wrote an application back in the 90s in VB6 that still is functional TODAY not just in Windows, but in Linux and MacOS (thanks to translation layers such as Wine)
The same can NOT be said for the game I made in C++/OpenGL later...
I have personally never regretted using tools that were dismissed as "toys" by the mainstream devs. Including HC, VB, and Arduino. I've got a VB5 program running in the factory that has had virtually zero failures in use for 14 years. I even hand-translated a curve fitting routine from Numerical Recipes into VB.
What angers me on Windows world is how the WinDev team routinely sabotages VB and C# efforts to be back to those productive flow in graphics programming on the platform.
Managed Direct X, XNA, all killed when their advocates gave up fighting for them.
Nowadays they proudly refer to Unity when one complains about missing .NET bindings to DirectX.
JavaScript's Standard Library is basically NodeJS + Browser WebAPI's.
UI libraries/toolkits were really only considered a part of any standard library in like one language, (Java) which is still a stretch, because even Seing and AWT were toolkits.
Standard libraries are small. That's the point. You use primitives to build up the rest.
"No standard library" might be too strong a criticism, but "a dozen or so built-in global variables with various methods" is still probably not the user experience that most desire.
Reminds of my time doing WebObjects development, and I fully agree that it seems like the frameworks I use today ( ReactJS, FeathersJS) aren’t (yet?) as simple to visually develop.
Great post - the one word missing, "maintenance." Most programming work is done maintaining/enhancing existing code. The greenfield work is a piece of cake by comparison.
You want to do the hard stuff? Maintain existing code you're not familiar with.
The problems around maintaining unfamiliar code are huge, largely unsolved, expensive and risky. There's a little branch of computer science called Program Comprehension and no one pays any attention. Though most programming money is spent on maintenance.
I always tell my clients - the difference between writing code and maintaining it, is the difference between raising your hands and keeping them raised indefinitely.
I sincerely don't know why uni don't make more classes on that only.
- pick any software
- try to change something
- give a precise impact analysis
- generate a few potential implementation paths
- measure how fast you did all that and what failed / worked
A fundamental problem with how we teach programming is that we focus on writing, instead of reading, software. To borrow terminology from the language arts; we don't focus enough on reading comprehension.
At the University I'm associated with, there was a discussion about why more 'practical' classes weren't taught. The answer was 'We're not a trade school.'
I've also wondered if it's time to back off on agile a bit. Or at least "agile" as it is implemented generally, which means we get to make up new requirements every two weeks. In my experience, the hardest maintenance problems occur because we're trying to re-shape code into something that the developers never knew would be coming down the line. Spending lots more time up-front deciding requirements would go a long ways towards more maintainable code.
It's interesting to me that people are so into Agile when it simply doesn't work all by itself. How many books, conferences, certifications, practitioners, evangelists, etc. does it take to make a paradigm, supposedly the paradigm, of working actually work? Every company I have worked for that used Agile basically had broken processes and were not productive. The one job where we did not explicitly use Agile was actually a place where I produced the most useful work.
If the number one answer to we're using Agile but it's not working is "you're doing Agile wrong", then maybe Agile isn't the solution?
The way I worked at the place that did not have an explicit process was one of switching between agile and waterfall methods. Early on in the projects, the process was primarily waterfall, defining things up front, building prototypes, laying out the project, etc. Then once passed that stage, projects would enter into a more agile or iterative process. Then as new big features came in, back to waterfall and then switching to agile once that stabilized. This worked quite well.
Maybe I'm just an old curmudgeon, but it seems to me there's way too much emphasis on speed of development in general. Time to market seems to be more important than quality, robustness, security, performance, or any other concern.
Another thing that rubs me wrong is the recurring notion that we need to get rid of the text as a representation of code. I've yet to meet a mathematician who wants to get completely rid of formulas because coming up with a proof is slow and cumbersome.
I understand that the incentives are drastically different between academia and business, but perhaps we've gone too far in this particular direction. Perhaps it's okay to admit that programming isn't as easy as we all want it to be and it's okay to take the time to change things carefully instead of moving fast and breaking stuff.
The part of Agile a lot of people ignore is the phase when, after your code works, you spend as much time as it takes to make it well structured and readable to others.
the problem is that a large amount of code produced is to support the accidental complexity of the software (let's call it infrastructure code) while the value generated by it comes from its essential complexity, that is, the application domain, as cited by the article .
while Domain Driven Design and Clean Architecture are a first step in the right direction, languages and frameworks are still limited in supporting these ideas.
It's almost insane to think that once you have a modeled domain you need to replicate them in resolvers, types, queries, etc (for graphql), resources, responses, endpoints (for rest), etc, etc.
to reinforce the point of the article and the one brought by @SKILNER, I believe that the great transformation that is to come is in maintainability through a focus on the domain
I think, as OP alludes, ultimately explorability is the heart of the problem that stops people from writing code this way.
If you don't care about other people being able to read and explore your code without a lot of preperation, you can go full hog creating layers of DSLs and metaprogramming, and with enough dedication you can end up with all your real domain level business rules in one place separate from the "infrastructure"
But if you do this, you end up with a codebase that is hard for a newcomer to ask simple maintenance questions about like "What are all the places XYZ is called from?" or "What are all the places to write to ABC" etc
So an experienced developer learns to limit their metaprogramming so their code retains easy explorability at the expense of long term sustainability. Golang is kind of an epitome of this kind of thinking. Lisps are kind of the epitome of the opposite I guess.
This is what's behind the paradox where a good dev working alone or maybe with one very like minded person can produce a level of productivity you can't match again as you add developers, till you have way more developers. The dip represents the loss due to having to communicate a common understanding of the codebase, that doesn't get adequately compensated for till you've added a lot more people.
I would argue that a huge or maybe even the primary reason why maintenance is so hard, though, is because a lot of software was originally not written "correctly", that is without maintenance and longevity in mind.
The hardest part of maintenance, in my experience, is that programs were developed under needless or incorrect constraints and then expected to be magically maintained. There is a huge downstream effect of decisions made early on in a software system's life.
The story of "just get it working" that evolves into "now that's it's working, don't change it but add these new features" repeats itself over, and over, and over. It's not surprising why maintenance in systems developed like that is hard.
yes. I wish I could somehow get to work on a blend between: a decompiler, debugger, emulator, static analyzer, memory profiler, and so on.
The idea being some kind of a runtime for assembly code which does not actually execute the program but allows one to understand it in different sematinc levels, or dunno.. this is a very raw idea. needs a lot of work (and a lot more knowldedge) to set down.
too bad none of the professors that I was able to meet were really interested in this kind of thing
I think dynamic analysis is incredibly powerful and criminally underused in IDEs and other dev tools.
I have thought of an idea about 6 months ago that has been fermenting in my mind since then : what if (e.g.) a Python VM had a mode where it records all type info of all identifiers as it executed the code and persisted this info into a standard format, later when you open a .py file in an IDE, all type info of the objects defined or named in the file are pulled from the persistent record and crunched by the IDE and used to present a top-notch dev experience?
The traditional achilles's heel of static analysis and type systems is Turing Completeness, traditional answers range from trying and giving up (Java's Object or Kotlin's Any?), not bothering to try in the first place (Ruby, Python, etc...), very cleverly restricting what you can say so that you never or rarely run into the embarrassing problems (Haskell, Rust,...), and whatever the fuck C++'s type system is. The type-profiling approach suggests another answer entirely : what if we just execute the damn thing without regard for types, like we already do now for Python and the like, but record everything that happens so that later static analysis can do a whole ton of things it can't do from program text alone. You can have Turing-Complete types that way, you just can't have them immediately (as soon as you write the code) or completely (as there are always execution paths that aren't visited, which can change types of things you think you know, e.g. x = 1 ; if VERY_SPECIFIC_RARE_CONDITION : x = "Hello" ).
You can have incredibly specific and fine-grained types, like "Dict[String->int] WHERE 'foo' in Dict and Dict['bar'] == 42", which is peculiar subset of all string-int dictionaries that satisfy the WHERE clause. All of this would be "profiled" automatically from the runtime, you're already executing the code for free anyway. Essentially, type- checking and inference becomes a never-halting computation amortized over all executions of a program, producing incremental results along the way.
I have ahead of me some opportunity to at least have a go at this idea, but I'm not completely free to pursue it (others can veto the whole thing) and I'm not sure I have all the angles or the prerequisite knowledge necessary to dive in and make something that matters. If anyone of the good folks at JetBrains or VisualStudio or similar orgs are reading this : please steal this idea and make it far better than I can, or at least pass it to others if you don't have the time.
There have been interesting progress in this space from a different angle of attack. o11y (Akita, honeycomb, ebpf, prodfiler, ...) and "Learning from incidents" are two of these angle that have really built on this idea that "understanding what the system does in prod" matters far more than "writing it right the first time".
It is also the thinking we can see supporting a lot of the early devops movement.
Alot of popular languages make it really easy to write code, but hard to maintain it .
It's one of the reasons I like Rust so much,even for relatively high level code. It has one of the srictest type systems and std/library ecosystems of any language in existence, which tends to make code very robust and easy to refactor.
I often wish for a language that has a similarly strict type type/ecosystem but is higher level. Swift is quite close, but very Mac OS centric. Haskell has exceptions, which are often used for error handling, making code less robust (plus the ecosystem is small). Other languages like Ada/Spark or Idris are way too fringe...
>Alot of popular languages make it really easy to write code, but hard to maintain it .
The language you're writing in has next to no meaningful impact on long term maintenance. When you're trying to do software archeology on why a function even exists the types it has will not help you figure out it's there because between 1996 and 2002 there was a tax rebate on capex for rocket development.
You need human understandable documentation, not fancy language features. This is no where near as sexy so no one does documentation and we're in the middle of a digital dark age.
Also possibly including: install very old OS version, get libs and dependencies to some old version, and maybe update to a slightly newer compatible version, but not the latest. Create mock services of complex back end systems for test cases. Convert https connections to http for testing, or use unsupported TLS versions and protocols for some connections. Then you can get started!
Working on old but popular software is where legends are made. You need to be methodical about changes. How do you make changes in a million line+ codebase without breaking anything for millions of existing users? This challenge is reserved for the finest engineers on the planet. These are the people you want your desk near when you start your professional career as a programmer.
this is simpler than it sounds - the process trumps any engineering stardom. the process is king. the process is love, the process is life, quite literally. getting any change in there isn't so much fine engineering as it is wrestling with layers and layers of process, where every layer has been added due to a monumental f-up in the past. it's an environment where getting any change committed into a repository usually takes weeks, unless one of processes for sidestepping the process is invoked.
A huge opportunity here: ways to assist developers to figure out how existing code base works by generating code flow, class hierarchy, etc.
yes there are a few tools on the market, nothing really standout though, maybe it's for AI/ML to innovate in this field.
Linux kernel is well designed, in that I can add code relatively easy into its subsystems, it's those Object oriented or FP code base that bothers me the most, especially when they're large, often times they just made me feel hopeless, good tools desperately needed.
A similar concept is the maintenance of physical world machinery, it's also often overlooked, but a huge industry even compared to manufacturing the machinery it self
I’ve been programming for over 30 years, and never been as productive as before. Need to load and decode a jpeg/png, I grab stb_image.h. Need to decode an ogg file, libogg. Need to decompress, libz. Need to decode video, libavformat. Need physics, libbullet. Need truetype fonts, freetype. Need a GUI, Qt. Need SSL, libssl.
My day becomes selecting libraries, integrating them, and testing integration. My business code is < 20% of our entire codebase. And I’m much more productive in 2022 than in 1992.
> My day becomes selecting libraries, integrating them, and testing integration.
Funny, being an old, cranky, gray-no-beard, if I wanted to spend my days selecting libraries, integrating them, and testing integration, I would have become a digital electronics engineer.
Your point is well taken, it's just our modern truth. It's just not was attracted me to this world in the first place.
> Writing tests is time-consuming, usually doesn't scale, and it easily creates tight coupling with implementations
After thousands of years, humans still use double accounting in finance. This is because it's effective. Unsurprisingly, it's tightly coupled to the original transaction recording.
The constant assertion that tight coupling is bad, because it creates more work which can be error prone. This is a fundamental problem with how people discuss software testing. It's not a panacea, but a guardrail. Guardrails aren't indestructible. That isn't the point. Developers have an obsession with making things as easy as possible (be that simplifying or codifying) and this is not a domain where that makes for a better solution, all other qualities being equal.
I think "double accounting" is a good analogy, and if you look at it that way, some tests that look like the same thing written twice in a different way start to make more sense.
The problem is just with effectivity - it'd be fine if the time spent on test would be the same as time on code, but very often the effort on test is much bigger while not even covering all important cases. I think we need to look for methods to get a good case coverage without spending majority of our time writing tests.
The alternative is shipping your own, which most of the time is a) a bad idea since you don't have domain specific knowledge and b) the customer is not paying for you to reimplement the 10th jpeg parsing function.
What is your alternative then? Vendored-in dependencies with their ossified security vulnerabilities? Or figuring out homegrown code of dubious quality for the functionality that is not core business of the company/product?
> My day becomes selecting libraries, integrating them, and testing integration.
This is why I hate modern programming.
When was the last time you had a job where you could actually learn how to actually program something? The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library.
Which means the more time you spend at a regular job, the worse your brain will rot.
Just talk to any programmer at any company and you will notice that almost none of them know how to program anything. They just know how to stitch together libraries with some glue code.
> The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library.
They are right, even if you do not agree. Handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security.
Huge third party libraries like LibOgg are maintained by gigantic industry players. Chances are, they are more knowledgeable in that specific domain than you are and they can focus more developer time on what is just a small part in your business application.
I find this comment amusing because most tasks the GP mentions are so specialized that you wouldn't actually have time to learn them. If your complaint is left pad and similar libraries, that's not what GP comment is talking about.
And still you're typing repetitive text. My dad was punching cards and able to read code from punched tape. I grew up writing code on a home computer's display. And now, 35 years later, it's long overdue that some visual NoCode tool should be the default for mostly repetitive programming jobs. If I want to build a Finder Quick Action to resize an image, for example, I certainly won't open a C editor, I'll just use Automator.
Eh, unfortunately the one programming breakthrough the world actually needs is one that would drastically change, and perhaps harm, most of the people around here.
We need more "Excels." More and better tools that let "regular" people program.
I'm a firm believer in empowering end user automation. The atrocities that folks cobble together with the Excels of the world are a marvel, and more power to them.
But in the end, we all talk about leaky abstractions, and the stark horror is that the fact that all of these wonders run on a computer, the worlds leakiest abstraction, the worlds most stubborn, pig headed, cantankerous contraption out there.
I think it goes beyond automation and into customization. Giving end users the power to "program the program" just makes it possible to make the thing do what they want in a way that it wasn't set out to do. Without having to pay/find/wait for a developer to do it. Sometimes it's about automation. Sometimes it isn't.
>More and better tools that let "regular" people program.
There's one catch with Excel, the more complicated your spreadsheet gets, the closer it gets to programming. And at some point, it's more cost effective to write in a proper programming language than continue to maintain mess.
However, until this complexity level, Excel is great, and empowers a lot of people who barely know the basic features of Excel to solve their computational problems.
You can already have "regular" people as software engineers if you follow basic concepts like the SOLID principles. Unfortunately many engineers are incredibly smart and they don't see the difference between writing spaghetti code with cryptic names and writing clear, legible code that normal people can follow. I've worked with people that could figure out a program written in binary if they needed to, but that meant that they never learned to write structured, sensible, readable programs.
Or more realistically the rapid application development tools need to be better so we're not bogged down dealing with latest fad or resume driven development
There's not much reason to improve things. No one's out here saying "you don't get this job because you chose new technologies when they weren't needed".
Even if you can equate glue code with trash, we seem to have unlimited space or a black hole to throw that trash.
It is not a fantasy for small businesses, who rely on internal tools and processes to operate. These tools solve real pain points for them and that is why they exist and why it is a fast growing market.
>We need more "Excels." More and better tools that let "regular" people program.
The only reason why more people don't program is because Windows is antithetical to programming _anything_. The second you remove people from a windows environment is the moment they start coding, even by accident.
I was agreeing with you, but as I thought about it, I remembered MS Access. It is used, often ships with Office, but still isn't close to as popular as Excel. Either the model is wrong, or the need isn't there.
In my experience, the main problem is that while Excel "distributed code versioning" with a whole bunch of copies around is awful, Access databases are not only much more prone to corruption and data loss in general (even in single user, non-simultaneous ideal use cases, never mind people attempting to multiplayer edit it over network shares...), it's often very hard for non-programmers to get any insights on any data you threw on it without using Excel in the first place.
So if the data is coming from Excel, and being extracted back to Excel, why use anything besides Excel?
Access used to be pretty standard for a small business trying to improve their situation, but we in the software development world spent so much time trying to work around limitations of Access and upgrading growing businesses out of Access that we took it personally and shouted to everyone not to use Access.
However, it is still taught in IT classes as a useful tool for small businesses, and it is.
Many more such cases [0], but Excel and spreadsheets in general also save businesses $$$$$, which explains their pervasiveness. Calculations take a long time and are easier for a human to mess up, which is why the initial spreadsheets were so popular as to drive the adoption of the personal computer in general [1]. The gulf between Excel and Python is pretty obvious, but there recently there have been few with their sights set on it, as most programmers looking to improve programming do so from their own experienced vantage point.
The question is not just how much Excel costs, it is how much revenue it helps bring in. And in many places, it brings more than it costs. Nothing is worse than a poorly designed webapp that was supposed to solve the "Excel problem" in a process.
Why would it harm us? We are problem solvers, we can use our intellect somewhere else. Programming is just a tool.
A lot of my colleagues are hybrid of traders + programmers. They do a lot of Excel. Sure, we can automate text programming, let everyone do Excel, and us problem solvers still won’t find difficulties finding high paying job.
True, but just using Excel does not lead to decent data models, though. IMHO, what we would need is for "regular" folks to grasp the basics of relational database first (at least 1:n and n:1) and then build an easy to use excel-like tool around it.
I'm not going to argue whether the 'world actually needs' that, or if people mostly, actually want that, or if, since excel is already there if you want it, you probably don't have to write, sell or build any more low code tools.
...because, although I could argue about those points, it's fundamentally unrelated to the OP, and issue that programming is hard, and, for a very long time, no one has really had any idea how to solve it.
Now, however, AI generated code (like copilot) is, for the first time in a long time, a potential avenue to actually change how software is created at all levels.
I think there's pretty interesting, because it opens up a lot of new opportunities.
Is the future dynamic languages / high level specifications that are AI-transformed into typed verbose languages like C/Rust/whatever and then compiled?
The ancient tools like Rational Rose tried to do this, but the tech was never actually technically good enough. Maybe... we'll see a real change in this space as the sophistication of the models improves... or maybe, like self-driving cars, it's always going to be 'nearly good enough'.
Hard to say.
...but, hot damn. More excels? Please no. Excel already exists. Don't rebuild that stuff again. Build new, different interesting tools please.
>...but, hot damn. More excels? Please no. Excel already exists. Don't rebuild that stuff again.
I don't think they literally meant another Excel. Just something that is easily accessible and usable by non-programmers to do very 'programmy' things, etc.
They mean no-code or low-code tools, but those come with a large hidden layer. These tools are known for making the 80% easy and the other 20% impossible. Excel can also become a mess.
>Is the future dynamic languages / high level specifications that are AI-transformed into typed verbose languages like C/Rust/whatever and then compiled?
How is that different from the present? Why are these "AI-transformations" different from what a compiler can do?
Yeah, I was using Excel as shorthand for "simple enough yet powerful enough tools such that someone who is perhaps an expert in something else can build themselves something useful."
Glue code is used to piece together libraries. I think it's a sign you used the right libraries and aren't reinventing. It's boring, but glue code essentially describes why your project is not the same as every other apo using those libraries.
Boilerplate is a thing that happens when libraries don't have sane defaults. It's one of the more unpleasant things about Docker et al. I wish we had a dev climate where something like Linux Standard Base could take off.
I find frameworks to be perfectly fine, as long as you break out of the "Perfectly fit the code to the vision in my head" model and go for "How do I make this product using only these high level blocks I have, without modifying them".
Frameworks are great as long as you don't fight them. Using a distro like Mint? Don't try to swap out core system services. Using a declarative web framework? Don't try to do some synchronous imperative stuff it was never meant to handle.
I get the feeling a lot of programmers really value creative freedom and don't want to feel like they are playing a rail shooter, just coding the one obvious thing the opinionated tools tell them to do, and then they have a bad time when they use tools that were specifically designed to remove any need for interesting code.
A lot of why we have containers is to stop dependency hell, but having a stable platform to built on is another way to achieve that without containers at all.
Of course it seems like the other reason we use them is to bundle multiple services together, but I'm not exactly a microservice fan to begin with.
In any case, I don't see why we couldn't have an OS that makes debian packages into containers, sharing things that are needed by multiple containers as appropriate while allowing overrides.
Android apps have a pretty nice model. If Android were a bit more open and had a few modifications to better support that use case, it seems like Dart/Android would make a pretty nice backend, assuming the ecosystem was there.
Android proved the benefits of an extremely opinionated inside the box kind of system, making APKs run on Linux and Windows seems like it would have a ton of uses.
I feel like a lot of boilerplate is a choice, that often comes from frameworks being used.
I disagree with the text criticism as well. The reason we spend time thinking and reasoning about code, is not because its text format. Its because that's the job. We aren't scribes just copying books. The hard part of programming is figuring out what to do, not writing it out. Well i am sure better visualizations or something could offer minor improvements to comprehending code bases, i doubt it will change things significantly. We will always spend most of our time trying to figure out what we need to do not doing it.
Anything that feels repetitive, like writing boiler plate code, is something that can, should, and probably will be automated eventually. This is a constant if you are a programmer, whatever you are doing today that is repetitive, you'll likely be using some better way of doing that in the future. And as Alan Kay says, the best way to invent the future is to invent it. I'm always on the lookout for doing the same things with less repetition.
Boiler plate usually results from people using something that was not originally designed to do that thing or not designed that well. It's indicative of some kind of design friction or feature creep. When it works, people just do more of it without really thinking about what they are doing or why. If you see people copy pasting the same blobs of code over and over again, that's a good sign something is wrong.
At some point that becomes the way things are done and people start nit picking each other about doing it properly and then inevitably somebody comes along and does a thing that is vastly simpler and accomplishes the same thing. Happens over and over again. Some people then inevitably resist that new way of doing things because they are really invested in the old way. But the way our industry works is kind of Darwinist; so those things tend to die out quickly once something better comes along.
The two mistakes people in this industry make over and over again is assuming that 1) they know it all and 2) things don't change. Because things actually do change, and usually for good reasons, the former requires work to stay true. Some skills last longer than others and not all change is great. If you are just coasting and writing the same stupid code over and over again like it's ground hog day, it's going to eventually run out on you.
> I feel like a lot of boilerplate is a choice, that often comes from frameworks being used. [...] I disagree with the text criticism as well.
The representation as text files is also part of what's causing the boilerplate.
As an example: How would you reduce the devops and configuration boilerplate in a monorepo?
In your typical JS microservice, a lot of code is not actually JS but yaml, json, terraform, etc. And it is very hard to abstract these away since a lot of tools rely on the existence of actual files.
Of course you can use code generation and other macros to manage the units in your monorepo (nx.js does this). But this is very instable since you might need to tweak some file resulting in your macros not working any more.
My take would be that textual representation is a local maximum. And our needs have outgrown this maximum. We're spending 90% of our time to innovate to push the boulder up the remaining 10% of this very hill. But there are most likely other hills with a much higher peak.
>In your typical JS microservice, a lot of code is not actually JS but yaml, json, terraform, etc. And it is very hard to abstract these away since a lot of tools rely on the existence of actual files.
This is a problem of your own making, though. If you don't do a microservice architecture in JS and do a modularized monolith in Go, as an example, you don't have to write any of this code.
> Maybe you could write tests as queries that would test a whole set of possible programs, not only the current version of your program at the moment.
I think that the future of programming is more sophisticated static analysis. Programmers will write statements like, "every code path that writes to the Payments database must have called validate_user()." Then, the tooling will confirm that rule with every commit.
We kind of have this already (for example, Facebook's Infer tool [0]), but I think it will become much more important in the coming decade.
To set up an example, Azure has some API management stuff that could let you do this before you even got to your code. Writing a tool to make sure that API management rule exists would be different than static analysis.
I'd agree with you, but not all of the business logic like that is going to live in code in the future.
I would love to see a language that considers tooling a first class problem. Most (all?) languages barely consider complex projects with 100s of dependencies and large multi-step builds. Java shouldn't have to wait for Maven and Gradle to come along, and Python shouldn't be shackled to piss-poor systems like pip.
This is something Rails gets very right. Bundler and Rake make Rails one of the nicest development environments there is.
But we can and should do better. Why is DLL-hell still a thing? Imports should specify versions, defaulting to a default if none is specified. Languages barely even consider versions today, they are almost always hacked on via magic text strings in filenames.
Why do tools like Dependabot and Renovate need to parse a million dependency formats to figure out versions? The language runtime or build system should produce that as a first class output.
Deployment and documentation are other places most languages ignore and delegate to a variety of crappy external tools.
Build Server Protocol is a good step in the right direction, but it barely scratches the surface. It would be great to see a language that prioritizes the parts of the developer experience that aren't writing code.
>But we can and should do better. Why is DLL-hell still a thing? Imports should specify versions, defaulting to a default if none is specified. Languages barely even consider versions today, they are almost always hacked on via magic text strings in filenames.
So suppose you fix the version number when you import the library. You then need to copy paste the version number to other imports in your code base? Or maybe you mean each import may have different versions? which is all right until you need to pass some data between two versions of the same library. This wouldn't be so difficult if semantic versioning or another equivalent was respected, but we know it's a lie. This wouldn't be a problem at all if there were some kind of contracts in place when using modules/package/whatever instead of a silly number based on convention but how to specify the contracts is not that easy, and it's similar to say having static typing. I think improving versioning it's doable but it's very hard.
I'm all about improving tooling, I have been obsessing for a few months about the sad state of program documentation and navigation. Why is the computer doing nothing while I think so hard while I'm programming? Why there is so little money in this space?
Go and Rust with go and cargo commands correspondingly have great tooling in this respect. As long as you use one language, everything is very convenient.
Readit News
Just to cherry pick a dramatized example. In the late 90s, one of my first programming experiences was in Delphi. It's a very visual way to program. You drag and drop a UI together from standardized elements. You could data bind things like input fields to a data source. You'd double click a button and it takes you to the already created "onclick" event where you do the actual coding. In a way, this is still similar to how you can "program" Microsoft Access today.
In this same period (I was in a computer science study) I was moaning to my teacher how tricky programming in C was. I made tiny memory management errors crashing the entire PC, with zero debug output.
His words: "Don't worry. By the time you get to work, you'll just do modeling".
Now I work on the web. It's been 30 years since Delphi and instead of productivity gains, we've taken steps in the opposite direction. There is no standardized UI library. JavaScript has no standard library. There's no single unified architecture. There's no standard tool chain. It's all very low level after which we stitch together fragile semi-abstractions.
I know it's an imperfect comparison, but I hope it gets the point across. It's all far too low level and fragile.
This doesn't just hurt productivity, also accessibility. It's simply not beginner friendly.
I do admit, though, that front-end seems like a wild west presently, to my ignorant eyes.
I really agree with this. The situation is a bit crazy. Since formal proofs are hard, the alternative taken is giving no guarantees at all. That's bad. Static analyzers can get really far. Contracts, even just at runtime are really useful. I don't want a piece of code to run if the precondition is not met, and I don't want it to return or have side effects if the postcondition is not met. This would rule out a massive class of errors. Of course, bonus points if you can do it at compile time.
Ada SPARK, Infer [1] and Dafny [2] show that rigorous engineering is possible and practical. Actually, Facebook (Meta now) argues Infer let's them move faster.
I think the problem is political and economical. It's hard to monetize this first. I wish someone like Apple took the plunge and rebuilt a simpler version of their stack using formal verification. It's the kind of thing only a large corporation or a government can do because it might take half a decade to take off. Once it takes off its unstoppable. Who would not like to buy e.g. a phone where most bugs and security breaches are impossible by construction? Verification also has a nice side effect. Change for the sake of change and featuritis are pretty incompatible with it.
[1] https://fbinfer.com
[2] https://github.com/dafny-lang/dafny
Yes! I remember Delphi as my first language in school, too. And later in university I could not understand, why in all the other advanced languages like Java, C and C++ doing the same was so hard.
Then I discovered Flash and later Flex with Flex Builder. That was a even way more powerful experience. You could draw up anything - and make a button out of it with a click. Or a sophisticated, reusable component.
And then it died, because of Adobes unwillingness to open source the technology (and their focus on performance and flashy features and not security). All the bad flash applets that were flying around existed, not because it was so bad, but because it was so easy even for total beginners to make something working fast.
But today? I am back to hacking scripts.
Regarding Java and C#, I am stunned by how good are IDEs now. To be clear, when I say IDE, I am specifically talking about a tool that enhances the edit-compile-debug cycle. Compared to 20 years ago when I wrote C, C++, and Perl, the IDEs were mostly terrible, except Visual Studio for Windows. The majority of my development cycle was all terminal-based: vim to edit, make to build (C/C++ only), gdb/perl -d to debug. Hoi, awful. (And, I still enjoy those all three languages!) As mega-corp computing slowly migrated to Java & C#, my productivity has increased dramatically. I guess at least 2x, if not more. Yes, some of this comes from how forgiving a modern VM can be (NullPointerExceptions, debugging, etc.), but a lot can ascribed to the incredible ecosystem of IDEs.
Regarding JavaScript, in the context of a modern web browser, high school students (in almost any country in the world) can easily Google a little bit, read some blogs, and then start GUI programming. And this doesn't require expensive, licensed software, like Delphi 30 years ago.
Delphi wasn't expensive for most of it's life, until they got greedy, and lost the market.
Lazarus is the modern open source replacement, it's mostly compatible with Delphi, and $1400 cheaper. ;-)
To prepare for that I got a copy of a version of Delphi for free on a magazine CD. Was still more productive than many tools now.
Blueprints are nicer for known problems, they are a better glue for old knowledge, but when you want to do something custom or new, you're out of luck.
This is the same problem with all programming, you can't make something specific and new without doing an engine teardown and rebuilding it. All the parts are terminally interconnected and always will be.
At the end of the day, UE4/5 is a landing pad for all the work you make in other tools. I agree web dev is no fun though.
One can pay for a CRM like Sitecore, AEM, low code like Outsystems, or enjoy the freedom of manually glueing HTML, JavaScript and JSS.
The most productive tools I used prior to learning all of that? HyperCard on the Mac, Visual Basic on Windows. Why? Tooling. Even when you had to get down and dirty, it seemed the tooling helped you out in some way. Shoot, I wrote an application back in the 90s in VB6 that still is functional TODAY not just in Windows, but in Linux and MacOS (thanks to translation layers such as Wine)
The same can NOT be said for the game I made in C++/OpenGL later...
Managed Direct X, XNA, all killed when their advocates gave up fighting for them.
Nowadays they proudly refer to Unity when one complains about missing .NET bindings to DirectX.
UI libraries/toolkits were really only considered a part of any standard library in like one language, (Java) which is still a stretch, because even Seing and AWT were toolkits.
Standard libraries are small. That's the point. You use primitives to build up the rest.
Deleted Comment
You want to do the hard stuff? Maintain existing code you're not familiar with.
The problems around maintaining unfamiliar code are huge, largely unsolved, expensive and risky. There's a little branch of computer science called Program Comprehension and no one pays any attention. Though most programming money is spent on maintenance.
Deleted Comment
A fundamental problem with how we teach programming is that we focus on writing, instead of reading, software. To borrow terminology from the language arts; we don't focus enough on reading comprehension.
If the number one answer to we're using Agile but it's not working is "you're doing Agile wrong", then maybe Agile isn't the solution?
The way I worked at the place that did not have an explicit process was one of switching between agile and waterfall methods. Early on in the projects, the process was primarily waterfall, defining things up front, building prototypes, laying out the project, etc. Then once passed that stage, projects would enter into a more agile or iterative process. Then as new big features came in, back to waterfall and then switching to agile once that stabilized. This worked quite well.
Another thing that rubs me wrong is the recurring notion that we need to get rid of the text as a representation of code. I've yet to meet a mathematician who wants to get completely rid of formulas because coming up with a proof is slow and cumbersome.
I understand that the incentives are drastically different between academia and business, but perhaps we've gone too far in this particular direction. Perhaps it's okay to admit that programming isn't as easy as we all want it to be and it's okay to take the time to change things carefully instead of moving fast and breaking stuff.
Not to mention massively underestimated and unappreciated by non-programmers.
I'm not sure if we are respected for this capability. Do people want to pay for what they do not understand why it is so hard?
while Domain Driven Design and Clean Architecture are a first step in the right direction, languages and frameworks are still limited in supporting these ideas.
It's almost insane to think that once you have a modeled domain you need to replicate them in resolvers, types, queries, etc (for graphql), resources, responses, endpoints (for rest), etc, etc.
to reinforce the point of the article and the one brought by @SKILNER, I believe that the great transformation that is to come is in maintainability through a focus on the domain
If you don't care about other people being able to read and explore your code without a lot of preperation, you can go full hog creating layers of DSLs and metaprogramming, and with enough dedication you can end up with all your real domain level business rules in one place separate from the "infrastructure"
But if you do this, you end up with a codebase that is hard for a newcomer to ask simple maintenance questions about like "What are all the places XYZ is called from?" or "What are all the places to write to ABC" etc
So an experienced developer learns to limit their metaprogramming so their code retains easy explorability at the expense of long term sustainability. Golang is kind of an epitome of this kind of thinking. Lisps are kind of the epitome of the opposite I guess.
This is what's behind the paradox where a good dev working alone or maybe with one very like minded person can produce a level of productivity you can't match again as you add developers, till you have way more developers. The dip represents the loss due to having to communicate a common understanding of the codebase, that doesn't get adequately compensated for till you've added a lot more people.
Dead Comment
The hardest part of maintenance, in my experience, is that programs were developed under needless or incorrect constraints and then expected to be magically maintained. There is a huge downstream effect of decisions made early on in a software system's life.
The story of "just get it working" that evolves into "now that's it's working, don't change it but add these new features" repeats itself over, and over, and over. It's not surprising why maintenance in systems developed like that is hard.
The idea being some kind of a runtime for assembly code which does not actually execute the program but allows one to understand it in different sematinc levels, or dunno.. this is a very raw idea. needs a lot of work (and a lot more knowldedge) to set down.
too bad none of the professors that I was able to meet were really interested in this kind of thing
I have thought of an idea about 6 months ago that has been fermenting in my mind since then : what if (e.g.) a Python VM had a mode where it records all type info of all identifiers as it executed the code and persisted this info into a standard format, later when you open a .py file in an IDE, all type info of the objects defined or named in the file are pulled from the persistent record and crunched by the IDE and used to present a top-notch dev experience?
The traditional achilles's heel of static analysis and type systems is Turing Completeness, traditional answers range from trying and giving up (Java's Object or Kotlin's Any?), not bothering to try in the first place (Ruby, Python, etc...), very cleverly restricting what you can say so that you never or rarely run into the embarrassing problems (Haskell, Rust,...), and whatever the fuck C++'s type system is. The type-profiling approach suggests another answer entirely : what if we just execute the damn thing without regard for types, like we already do now for Python and the like, but record everything that happens so that later static analysis can do a whole ton of things it can't do from program text alone. You can have Turing-Complete types that way, you just can't have them immediately (as soon as you write the code) or completely (as there are always execution paths that aren't visited, which can change types of things you think you know, e.g. x = 1 ; if VERY_SPECIFIC_RARE_CONDITION : x = "Hello" ).
You can have incredibly specific and fine-grained types, like "Dict[String->int] WHERE 'foo' in Dict and Dict['bar'] == 42", which is peculiar subset of all string-int dictionaries that satisfy the WHERE clause. All of this would be "profiled" automatically from the runtime, you're already executing the code for free anyway. Essentially, type- checking and inference becomes a never-halting computation amortized over all executions of a program, producing incremental results along the way.
I have ahead of me some opportunity to at least have a go at this idea, but I'm not completely free to pursue it (others can veto the whole thing) and I'm not sure I have all the angles or the prerequisite knowledge necessary to dive in and make something that matters. If anyone of the good folks at JetBrains or VisualStudio or similar orgs are reading this : please steal this idea and make it far better than I can, or at least pass it to others if you don't have the time.
It is also the thinking we can see supporting a lot of the early devops movement.
Alot of popular languages make it really easy to write code, but hard to maintain it .
It's one of the reasons I like Rust so much,even for relatively high level code. It has one of the srictest type systems and std/library ecosystems of any language in existence, which tends to make code very robust and easy to refactor.
I often wish for a language that has a similarly strict type type/ecosystem but is higher level. Swift is quite close, but very Mac OS centric. Haskell has exceptions, which are often used for error handling, making code less robust (plus the ecosystem is small). Other languages like Ada/Spark or Idris are way too fringe...
The language you're writing in has next to no meaningful impact on long term maintenance. When you're trying to do software archeology on why a function even exists the types it has will not help you figure out it's there because between 1996 and 2002 there was a tax rebate on capex for rocket development.
You need human understandable documentation, not fancy language features. This is no where near as sexy so no one does documentation and we're in the middle of a digital dark age.
Deleted Comment
yes there are a few tools on the market, nothing really standout though, maybe it's for AI/ML to innovate in this field.
Linux kernel is well designed, in that I can add code relatively easy into its subsystems, it's those Object oriented or FP code base that bothers me the most, especially when they're large, often times they just made me feel hopeless, good tools desperately needed.
My day becomes selecting libraries, integrating them, and testing integration. My business code is < 20% of our entire codebase. And I’m much more productive in 2022 than in 1992.
Funny, being an old, cranky, gray-no-beard, if I wanted to spend my days selecting libraries, integrating them, and testing integration, I would have become a digital electronics engineer.
Your point is well taken, it's just our modern truth. It's just not was attracted me to this world in the first place.
After thousands of years, humans still use double accounting in finance. This is because it's effective. Unsurprisingly, it's tightly coupled to the original transaction recording.
The constant assertion that tight coupling is bad, because it creates more work which can be error prone. This is a fundamental problem with how people discuss software testing. It's not a panacea, but a guardrail. Guardrails aren't indestructible. That isn't the point. Developers have an obsession with making things as easy as possible (be that simplifying or codifying) and this is not a domain where that makes for a better solution, all other qualities being equal.
The problem is just with effectivity - it'd be fine if the time spent on test would be the same as time on code, but very often the effort on test is much bigger while not even covering all important cases. I think we need to look for methods to get a good case coverage without spending majority of our time writing tests.
It feels anything but productive.
This is why I hate modern programming.
When was the last time you had a job where you could actually learn how to actually program something? The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library.
Which means the more time you spend at a regular job, the worse your brain will rot.
Just talk to any programmer at any company and you will notice that almost none of them know how to program anything. They just know how to stitch together libraries with some glue code.
They are right, even if you do not agree. Handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security.
Huge third party libraries like LibOgg are maintained by gigantic industry players. Chances are, they are more knowledgeable in that specific domain than you are and they can focus more developer time on what is just a small part in your business application.
We need more "Excels." More and better tools that let "regular" people program.
The main problem is that MOST "programming" languages/environments are not made for business apps, (or for other niches, btw).
Python? Ruby? Julia? Delphi? all fail to work to make(or solve even small cases of) business apps.
By a long mile.
-----
What Excel solves, is have a barebones, half-working, unscalable, ill-suited... BUT MADE FOR help people get a "solution" for this.
Things that are more like this are Django, Jupyter notebooks, a little of Julia, FoxPro. But still are more pro-developer than pro-power-user.
I'd say the great weakness of No Code solutions isn't so much they are bad, but that they have no FOSS support. At all.
But in the end, we all talk about leaky abstractions, and the stark horror is that the fact that all of these wonders run on a computer, the worlds leakiest abstraction, the worlds most stubborn, pig headed, cantankerous contraption out there.
And all the lipstick in the world can't hide it.
There's one catch with Excel, the more complicated your spreadsheet gets, the closer it gets to programming. And at some point, it's more cost effective to write in a proper programming language than continue to maintain mess.
However, until this complexity level, Excel is great, and empowers a lot of people who barely know the basic features of Excel to solve their computational problems.
Even if you can equate glue code with trash, we seem to have unlimited space or a black hole to throw that trash.
I'm all into PowerApps and all that, but no-code for anything the least bit sizable or complex is a fantasy from citizen developers.
The only reason why more people don't program is because Windows is antithetical to programming _anything_. The second you remove people from a windows environment is the moment they start coding, even by accident.
If MS prioritised working on it it may have helped lift some people out of Excel and into DBA-lite territory but not in its current state.
So if the data is coming from Excel, and being extracted back to Excel, why use anything besides Excel?
However, it is still taught in IT classes as a useful tool for small businesses, and it is.
[0] http://www.eusprig.org/horror-stories.htm
[1] https://www.wired.com/2014/10/a-spreadsheet-way-of-knowledge...
A lot of my colleagues are hybrid of traders + programmers. They do a lot of Excel. Sure, we can automate text programming, let everyone do Excel, and us problem solvers still won’t find difficulties finding high paying job.
Deleted Comment
https://www.microsoft.com/en-us/research/podcast/advancing-e...
User defined functions:
https://www.microsoft.com/en-us/research/blog/lambda-the-ult...
...because, although I could argue about those points, it's fundamentally unrelated to the OP, and issue that programming is hard, and, for a very long time, no one has really had any idea how to solve it.
Now, however, AI generated code (like copilot) is, for the first time in a long time, a potential avenue to actually change how software is created at all levels.
I think there's pretty interesting, because it opens up a lot of new opportunities.
Is the future dynamic languages / high level specifications that are AI-transformed into typed verbose languages like C/Rust/whatever and then compiled?
The ancient tools like Rational Rose tried to do this, but the tech was never actually technically good enough. Maybe... we'll see a real change in this space as the sophistication of the models improves... or maybe, like self-driving cars, it's always going to be 'nearly good enough'.
Hard to say.
...but, hot damn. More excels? Please no. Excel already exists. Don't rebuild that stuff again. Build new, different interesting tools please.
I don't think they literally meant another Excel. Just something that is easily accessible and usable by non-programmers to do very 'programmy' things, etc.
How is that different from the present? Why are these "AI-transformations" different from what a compiler can do?
Boilerplate is a thing that happens when libraries don't have sane defaults. It's one of the more unpleasant things about Docker et al. I wish we had a dev climate where something like Linux Standard Base could take off.
I find frameworks to be perfectly fine, as long as you break out of the "Perfectly fit the code to the vision in my head" model and go for "How do I make this product using only these high level blocks I have, without modifying them".
Frameworks are great as long as you don't fight them. Using a distro like Mint? Don't try to swap out core system services. Using a declarative web framework? Don't try to do some synchronous imperative stuff it was never meant to handle.
I get the feeling a lot of programmers really value creative freedom and don't want to feel like they are playing a rail shooter, just coding the one obvious thing the opinionated tools tell them to do, and then they have a bad time when they use tools that were specifically designed to remove any need for interesting code.
Of course it seems like the other reason we use them is to bundle multiple services together, but I'm not exactly a microservice fan to begin with.
In any case, I don't see why we couldn't have an OS that makes debian packages into containers, sharing things that are needed by multiple containers as appropriate while allowing overrides.
Android apps have a pretty nice model. If Android were a bit more open and had a few modifications to better support that use case, it seems like Dart/Android would make a pretty nice backend, assuming the ecosystem was there.
Android proved the benefits of an extremely opinionated inside the box kind of system, making APKs run on Linux and Windows seems like it would have a ton of uses.
I disagree with the text criticism as well. The reason we spend time thinking and reasoning about code, is not because its text format. Its because that's the job. We aren't scribes just copying books. The hard part of programming is figuring out what to do, not writing it out. Well i am sure better visualizations or something could offer minor improvements to comprehending code bases, i doubt it will change things significantly. We will always spend most of our time trying to figure out what we need to do not doing it.
Boiler plate usually results from people using something that was not originally designed to do that thing or not designed that well. It's indicative of some kind of design friction or feature creep. When it works, people just do more of it without really thinking about what they are doing or why. If you see people copy pasting the same blobs of code over and over again, that's a good sign something is wrong.
At some point that becomes the way things are done and people start nit picking each other about doing it properly and then inevitably somebody comes along and does a thing that is vastly simpler and accomplishes the same thing. Happens over and over again. Some people then inevitably resist that new way of doing things because they are really invested in the old way. But the way our industry works is kind of Darwinist; so those things tend to die out quickly once something better comes along.
The two mistakes people in this industry make over and over again is assuming that 1) they know it all and 2) things don't change. Because things actually do change, and usually for good reasons, the former requires work to stay true. Some skills last longer than others and not all change is great. If you are just coasting and writing the same stupid code over and over again like it's ground hog day, it's going to eventually run out on you.
The representation as text files is also part of what's causing the boilerplate.
As an example: How would you reduce the devops and configuration boilerplate in a monorepo?
In your typical JS microservice, a lot of code is not actually JS but yaml, json, terraform, etc. And it is very hard to abstract these away since a lot of tools rely on the existence of actual files.
Of course you can use code generation and other macros to manage the units in your monorepo (nx.js does this). But this is very instable since you might need to tweak some file resulting in your macros not working any more.
My take would be that textual representation is a local maximum. And our needs have outgrown this maximum. We're spending 90% of our time to innovate to push the boulder up the remaining 10% of this very hill. But there are most likely other hills with a much higher peak.
This is a problem of your own making, though. If you don't do a microservice architecture in JS and do a modularized monolith in Go, as an example, you don't have to write any of this code.
I think that the future of programming is more sophisticated static analysis. Programmers will write statements like, "every code path that writes to the Payments database must have called validate_user()." Then, the tooling will confirm that rule with every commit.
We kind of have this already (for example, Facebook's Infer tool [0]), but I think it will become much more important in the coming decade.
0: https://github.com/facebook/infer
I'd agree with you, but not all of the business logic like that is going to live in code in the future.
https://youtu.be/OyfBQmvr2Hc
It’s exploiting logic programming to query for a program that satisfies some condition.
This is something Rails gets very right. Bundler and Rake make Rails one of the nicest development environments there is.
But we can and should do better. Why is DLL-hell still a thing? Imports should specify versions, defaulting to a default if none is specified. Languages barely even consider versions today, they are almost always hacked on via magic text strings in filenames.
Why do tools like Dependabot and Renovate need to parse a million dependency formats to figure out versions? The language runtime or build system should produce that as a first class output.
Deployment and documentation are other places most languages ignore and delegate to a variety of crappy external tools.
Build Server Protocol is a good step in the right direction, but it barely scratches the surface. It would be great to see a language that prioritizes the parts of the developer experience that aren't writing code.
So suppose you fix the version number when you import the library. You then need to copy paste the version number to other imports in your code base? Or maybe you mean each import may have different versions? which is all right until you need to pass some data between two versions of the same library. This wouldn't be so difficult if semantic versioning or another equivalent was respected, but we know it's a lie. This wouldn't be a problem at all if there were some kind of contracts in place when using modules/package/whatever instead of a silly number based on convention but how to specify the contracts is not that easy, and it's similar to say having static typing. I think improving versioning it's doable but it's very hard.
I'm all about improving tooling, I have been obsessing for a few months about the sad state of program documentation and navigation. Why is the computer doing nothing while I think so hard while I'm programming? Why there is so little money in this space?
Most of the time the CPU is waiting for IO - memory, network, SSD, disk and not doing any work.
You might like my idea called GUI Thunking.
https://github.com/samsquire/gui-thunks
Dead Comment
Old languages also have gained good tooling, but IMO the problem is that they are not as well documented as in-built systems of newer languages.
Deleted Comment
Deleted Comment
Deleted Comment