I found one of the perceived weaknesses of Clojure (in this article), it being dynamically typed, is a tradeoff rather than a pure negative. But it applies that tradeoff differently than dynamic languages I know otherwise and that difference is qualitative: It enables a truly interactive way of development that keeps your mind in the code, while it is running. This is why people get addicted to Lisp, Smalltalk and similar languages.
> To understand a program you must become both the machine and the program.
- Epigrams in Programming, Alan Perlis
Two of the big advantages of (gradually-) typed languages are communication (documentation) and robustness. These can be gained back with clojure spec and other fantastic libraries like schema and malli. What you get here goes way beyond what a strict, static type systems gets you, such as arbitrary predicate validation, freely composable schemas, automated instrumentation and property testing. You simply do not have that in a static world. These are old ideas and I think one of the most notable ones would be Eiffel with it's Design by Contract method, where you communicate pre-/post-conditions and invariants clearly. It speaks to the power of Clojure (and Lisp in general) that those are just libraries, not external tools or compiler extensions.
In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
The current crop of statically typed languages (from the oldest ones, e.g. C#, to the more recent ones, e.g. Kotlin and Rust) is basically doing everything that dynamically typed languages used to have a monopoly on, but on top of that, they offer performance, automatic refactorings (pretty much impossible to achieve on dynamically typed languages without human supervision), fantastic IDE's and debuggability, stellar package management (still a nightmare in dynamic land), etc...
Yeah, I had a fairly large (about a year of solo dev work) app that I maintained both Clojure and F# ports of, doing a compare and contrast of the various language strengths. One day I refactored the F# to be async, a change that affected like half the codebase, but was completed pretty mechanically via changing the core lines, then following the red squigglies until everything compiled again, and it basically worked the first time. I then looked at doing the same to the Clojure code, poked at it a couple times, and that was pretty much the end of the Clojure port.
> In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
Only if you are skimping on tests. There's a tradeoff here - "dynamically typed" languages generally are way easier to write tests for. The expectation is that you will have plenty of them.
Given that most language's type systems are horrible (Java and C# included) I don't really think it's automatically a net gain. Haskell IS definitely a net gain, despite the friction. I'd argue that Rust is very positive too.
Performance is not dependent on the type system, it's more about language specification (some specs paint compilers into a corner) and compiler maturity. Heck, Javascript will smoke many statically typed languages and can approach even some C implementations(depending on the problem), due to the sheer amount of resources that got spent into JS VMs.
Some implementations will allow you to specify type hints which accomplish much of the same. Which is something you can do on Clojure by the way.
Automatic 'refactorings' is also something that's very language dependent. I'd argue that any Lisp-like language is way easier for machines to process than most "statically typed" languages. IDEs and debugability... have you ever used Common Lisp? I'll take a condition system over some IDE UI any day. Not to mention, there's less 'refactoring' needed.
Package management is completely unrelated to type systems.
Rust's robust package management has more to do with it being a modern implementation than with its type system. They have learned from other's mistakes.
Sure, in a _corporate_ setting, where you have little control over a project that spans hundreds of people, I think the trade-off is skewed towards the most strict implementation you can possibly think of. Not only type systems, but everything else, down to code standards (one of the reasons why I think Golang got popular).
In 2021, I would expect people to keep the distinction between languages and their implementations.
We can debate this forever, but all I can say is that at my work we have equal part Java and Clojure, and we have some Kotlin and sole Scala as well. Out of all of them, Clojure does not cause us anymore issues, it doesn't take us any longer to add features, it doesn't perform any worse, and it doesn't have any more defects than the others.
My conclusion is that it's a matter of personal preference honestly. Those are all really good languages. Personally I have more fun and enjoy using Clojure more. I would say I tend to find I'm more productive in it, but I believe that's more a result of me finding using it more enjoyable then anything else.
I don't pick Clojure for its dynamic typing, I pick it for other reasons. I've tried Haskell but it really doesn't seem to mesh with the way I tend to develop a program. But I would love to have more static languages with the pervasive immutability of Clojure.
It's your opinion though, there's nothing scientific about what you're saying.
Take mocking for example, in Ruby/Rails it's a breeze. In Java you need to invent a dependency injection framework (Spring) to do it.
Not only this, but the programming style where you pass around dictionaries / maps for everything yet have expectations about what keys they contain works just as easily in JS, and with TypeScript or Flow you get a lot more help from the compiler than you do using spec (as I understand it).
Although you are right, the Clojure community probably by and large agrees with you. That is why everyone is excited about spec - it looks a lot like a type system for Clojure.
Agreed. I feel Lisps and SmallTalk are dynamic done right. I think the other language features that you use also influence the value from dynamic or static types. For OOP style, static types are a huge asset for refactoring and laying our architecture. On the other hand, immutable data and stateless functions (as idiomatic in clojure) make them less necessary, and also work great together with interactive development.
Except, of course, that specs are only tested correct, not proven correct like types would be. Types (in a reasonable static type system, not, say, C) are never wrong. In addition, specs do not compose, do they ? If you call a function g in a function f, there is no automatic check that their specs align.
> Types (in a reasonable static type system, not, say, C) are never wrong.
Oh man. This is the fundamental disagreement. Sure, you can have a type system that is never wrong in its own little world. But, that's not the problem. A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all. It's like you got the wrong answer really, really right.
> Except, of course, that specs are only tested correct, not proven correct like types would be.
Yes this is the fundamental tradeoff. Specs et al are undoubtedly more flexible and expressive than static type systems, at the expense of some configurable error tolerance. I don't think one approach is generally better than the other, it's a question of tradeoffs between constraint complexity and confidence bounds.
Yeah, and I think this is obvious, but it certainly depends on the origin of the data being checked. We can prove the structure of “allowed” data ahead of time if we want guarantees on what’s possible inside our program. We also want a facility to check data encountered by the running program (i.e. from the user or another program.) which of course we can’t know ahead of time.
It is a design decision to be able to build a clojure system interactively while it is running, so a runtime type checker is a way for the developer to give up the safety of type constraints for this purpose—by using the same facility we already need in the real world, a way to check the structure of data we can’t anticipate.
Yes, I think that is one of the big weaknesses of it. You can write specs that make no sense and it will just let you. So far there is also no way to automatically check whether you are strengthening a guarantee or weaken your assumptions relative to a previous spec. In a perfect world we would have this in my opinion.
Not only that, Smalltalk and Lisps are languages designed with developer experience as part of the language.
You just don't get an interpreter/compiler and have to sort everything else by yourself, no, there is a full stack experience and development environment.
Well it does include that kind of behaviour but it's quite a bit more than just that. E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction. I'm not necesarily saying you should but just to give an illustrative example that there's less restrictions on your freedom to express what you need than in a static system.
>> types are isomorphic with schemas
I don't think that's a good way to think of this, you're imagining a rigid 1:1 tie of data and spec yet i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
Right! I made the dumb, typical error to write: "You simply do not have that in a static world." When I should have written: "This type of expressiveness is not available in mainstream statically typed languages".
With "freely composable" I mean that you can program with these schemas as they are just data structures and you only specify the things you want to specify. Both advantage and the disadvantage is that this is dynamic.
You mean implicit type conversions? That's a thing you can get somewhat used to. But it throws off beginners and can introduce super weird bugs, because they hide bugs in weird ways, even if you are more experienced. Yes, I find strong typing strictly better than weak typing.
An even better example of this would be Excel, the horror stories are almost incredible.
So even if your environment is dynamic, you want clarity when you made a mistake. Handling errors gracefully and hiding them are very different things. The optimal in a dynamic world is to facilitate reasoning while not restricting expression.
This is a consequence of weak typing rather than dynamic typing. I appreciate that these are not precise terms, but being able to change something's type (dynamic) is different to the language just doing strange things when you combine types (weak).
I've admittedly not played with spec, but can't you solve documenting interfaces by defining `defrecord`s ? You rarely really care about the actual types involved. You just want to know which fields you either need to provide or will recieve
Spec will give you stronger feedback than a docstring or function signature. It can tell you (in code terms, with a testable predicate) if a call to an interface wouldn't make sense.
Eg, spec can warn you when an argument doesn't make sense relative to the value of a second argument. Eg, with something like (modify-inventory {:shoes 2} :shoes -3) spec could pick up that you are about to subtract 3 from 2 and have negative shoes (impossible!) well before the function is called - so you can test elsewhere in the code using spec without having to call modify-inventory or implement specialist checking methods. And a library author can pass that information up the chain without clear English documentation and using only standard parts of the language.
You can't do that with defrecord, but it is effectively a form of documentation about how the arguments interact.
Maybe I've just never given it a chance, but I've never understood the appeal of being able to modify code in-memory while it's running.
I like a REPL for testing things out, or for doing quick one-off computations, but that's it. I would never want to, say, redefine a function in memory "while the code is running". Not just because of ergonomics, but because if I decide to keep that change, I now have to track down the code I typed in and manually copy it back over into my source files (assuming I can still find it at all). And if I make a series of changes over a session, the environment potentially gets more and more diverged from what's in sourced if I forget to copy any changes over. So I'd often want to re-load from scratch anyway, at least before I commit.
Am I missing something? Am I misunderstanding what people mean when they talk about coding from a REPL?
You can get an approximate taste of what it's like in plain old Java + JRebel, it's seriously about the same as trying to do it with Python + something like Flask. Start up a big application server in debug mode with Eclipse/IntelliJ, be annoyed that it takes 2+ minutes to start after everything's compiled. Now you want to work on a story, or some bug. You can make changes, save, it recompiles just that file (incremental compilation is a godsend in itself), and hotswaps it into the running application memory. No need to restart anything (with JRebel, for most common kinds of changes; without JRebel, only for some changes). It's particularly useful when you're debugging, you find the problem, change the code, and re-execute immediately to verify to yourself it's fixed.
You also can get an experience like PHP, where you just have to change and save some files, and your subsequent requests will use the new code. This is so much better than shutting down everything and restarting and is a large part of why CGI workflows dominated the web.
Common Lisp takes these experiences and dials them to 11, the whole language is built to reinforce the development style of dynamic changes, rather than an after-thought that requires a huge IDE+proprietary java agent. It's still best to use some sort of editor or IDE, and then you don't have any worry about source de-syncs -- frequently you'll make multiple changes across multiple files and then just reload the whole module and any files that changed with one function call, which you might bind to an editor shortcut, but crucially like debugging is not centrally a feature of the editor but the language; the language's plain REPL by itself is just a lot more supportive of interactive development than Python/JS/Ruby's. Clojure, and I personally think even Java with the appropriate tools, are between Python and CL for niceness of interactive development, but Clojure tends to be better than the Java IDE experience because of its other focus on immutability.
As others have mentioned - you’re probably talking about something like a python or node repl. Lisp repl development is not like python or node - you _work_ in the repl. The closer comparison might be between bash+node as a repl - up+enter and the like to rerun tests has an equivalent in clojure+IntelliJ. There’s no copy pasting but there are different key bindings.
One of the best parts about lisp style repl development is that you end up doing TDD automatically. You just redefine a function until it does what you want from sample data you pass in - without changing files or remembering how your test framework works. You can save the output of some http call in a top level variable and iterate on the code to process it into something useful. The code you evaluate lives in the file that will eventually house it anyway so it’s pretty common to just eval the entire file instead of just one function.
Since you don’t ever shut the repl down, developing huge apps is also quite pleasant. You only reload the code that you’re changing - not the rest of the infra so things like “memoize” can work in local development. That’s why it’s a bit closer to your bash shell in other languages.
If you’ve never tried it, I highly recommend trying the Clojure reloaded workflow [1] to build a web app with a db connection. You can really get into a flow building stuff instead of waiting for your migrations to run on every test boot.
Yes, what you are missing is that usually you are using an editor like emacs where you can modify a specific form and send it to the repl. That way there is no chasing back through repl history for a form to paste back into a file.
> These can be gained back with clojure spec and other fantastic libraries like schema and malli. What you get here goes way beyond what a strict, static type systems gets you
That reads like what someone would think when they used the typesystem of Java 6 and now think that this is what "statically typed programming" means.
No, you can _not_ get back what types give you by any kind of spec - if anything you can get some of the benefits of types, but you also pay a price.
The thing is - dynamically typed languages don't really seem to evolve anymore. They add a bit of syntatic sugar here and there and sometimes add some cool feature, but mostly only features that already existed for a long time in other languages. At least that is what I have seen over the past couple of years, I would be happy to be proven wrong.
Looking at statical typesystems however, there is much more progress, simply because they are much more complex and not as optimized. From row-types over implicits and context-expressions towards fully fledged value-dependent typesystems, which have amazing features that start to slowly trickle down into mainstream languages like Typescript or Scala.
While both dynamically and statically typed languages have their pros and cons and it will stay like that forever, I expect that statically typed languages will proceed to become the bigger and bigger piece of the cake, simply because they have more potential for optimizations going forward.
I keep seeing lisp people bandy about all of this design by contract/arbitrary predicate validation stuff. Can you give an example of an instance in which static types + runtime checks don't completely subsume this?
My intuition is that almost all of these methods people are talking about would have to be enforced at run-time, in which case I don't see how it's providing anything fundamentally more than writing an assertion or a conditional.
They don't make these impossible, they typically just don't let you express these within the type system and they typically don't let you not specify your types.
I should have made clear that I'm emphasizing the advantages of being dynamic to describe and check the shape of your data to the degree of your choosing. Static typing is very powerful and useful, but writing dynamic code interactively is not just "woopdiedoo" is kind of the point I wanted to make without being overzealous/ignorant.
I feel like there's a missing axis in the static/dynamic debate: the language's information model.
In an OOP language, types are hugely important, because the types let you know the object's ad-hoc API. OOP types are incredibly complicated.
In lisps, and Clojure in particular, your information model is scalars, lists, and maps. These are fully generic structures whose API is the standard Clojure lib. This means that its both far easier to keep the flow of data through your program in your head.
This gives you a 2x2 matrix to sort languages into, static vs dynamic, and OOP vs value based.
* OOP x static works thanks to awesome IDE tooling enabled by static typing
* value x static works due to powerful type systems
* value x dynamic works due to powerful generic APIs
* OOP x dynamic is a dumpster fire of trying to figure out what object you're dealing with at any given time (looking right at you Python and Ruby)
One thing I don't like about all articles on clojure is that basically all of them say: ah, it's just like lisp with lists `(an (example of) (a list))` with vectors `[1 2 3]` thrown in. So easy!
But then you get to Clojure proper, and you run into additional syntax that either convention or functions/macros that look like additional syntax.
Ok, granted, -> and ->> are easy to reason about (though they look like additional syntax).
But then there's entirely ungooglable ^ that I see in code from time to time. Or the convention (?) that call methods on Java code (?) with a `.-`
Or atoms defined with @ and dereferenced with *
Or the { :key value } structure
There's way more syntax (or things that can be perceived as syntax, especially to beginners) in Clojure than the articles pretend there is.
(defn ^:export db_with [db entities]
(d/db-with db (entities->clj entities)))
(defn entity-db
"Returns a db that entity was created from."
[^Entity entity]
{:pre [(de/entity? entity)]}
(.-db entity))
(defn ^:after-load ^:export refresh []
(let [mount (js/document.querySelector ".mount")
comp (if (editor.debug/debug?)
(editor.debug/ui editor)
(do
(when (nil? @*post)
(reset! *post (-> (.getAttribute mount "data") (edn/read-string))))
(editor *post)))]
(rum/mount comp mount)))
is calling the method `.getAttribute` on the `mount` object – since it's a Lisp, it's in prefix notation. It also highlights how methods are not special and just functions that receive the object as first argument.
Finally,
@*post
is the same as
(deref *post)
and the `*` means nothing to the language – any character is valid on symbol names, the author just chose an asterisk.
Most of what you believe to be syntax are convenience "reader macros" (https://clojure.org/reference/reader), and you can extend with your own. You can write the same code without any of it, but then you'll have more "redundant" parenthesis.
Minor point of order about the atoms: they're not defined with @ nor derefd with . If you're referring to earmuffs* that's convention not syntax (specifically for dynamically scoped variables, which could be atoms or anything else), and @ is indeed deref. (More specifically @x is a reader macro ish that expands to literally `(deref x)`.)
Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language. I’ve seen that happen a few times. That code is hard to read and understand. This is why clojure will remain niche.
You need a team that wants to use Clojure. I wrote Clojure professionally for 2 years, and everyone at the company was excited about it and sold on the language. Even after 3-5 years of programming in it. Now, at a different place, we write in a different language, and even though I still love Clojure, I'm not gonna write some project in it, even if Clojure might suit it so well, because I know these people are sold on different language, and I'm not going to preach and I'm not going to make their lives more difficult by having to maintain some obscure codebase.
It could have been Go and Java programmer trying to understand it. Or it could have been some clumsy tool written in node which Go programmer finds hard to read and understand. Clojure's main advantage is that you can you can learn it very very quickly up to the point when you understand most of the code, the language is very very small compared to "five main languages".
> Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language
"Normal language"?
You mean, whatever language is most popular at the company. What's "normal" at one would be completely alien at another. Even things like Java. If you don't have anything in the Java ecosystem, the oddball Java app will be alien and will likely get rewritten into something else.
The reason Clojure remains niche is that some people somehow think it's not a "normal" language, for whatever reason.
is it really hard to read (could be) or is it just that the average coder never saw lisp or sml and doesn't want to bother bearing the responsibility to learn something alien on duty ?
Agreed. These days I'm really fascinated by clojure and trying to learn clojure. Other than the project setup and repl and the editor (which I had considered), these weird characters are throwing me off.
What clojure really needs is some kind of opinionated framework or starter template, something like create-react-app. That has all these things figured out so a beginner like me can start playing with actual clojure, which documents all the steps to setup the repl and editor and what not. The last time I asked for this I was told about lein templates, they help but there's no documentation to go with those.
There needs to be some push from the top level. create-react-app was produced by facebook. Elm reactor (which lets you just create a .elm file and play with elm) was created by Evan the language creator himself.
tldr: There's a huge barrier to start playing with clojure that needs to come down and the push needs happen from the top level.
> An incoming HTTP request? it is a plain Clojure dictionary.
I learned to code in Python. Loved it. Dynamically typed dicts up the wazoo!
Then I learned why I prefer actual types. Because then when I read code, I don't have to read the code that populates the dicts to understand what fields exist.
The two are not mutually exclusive. Clojure has namespaced keywords and specs[0] to cover that. (There is also the third-party malli, which takes a slightly different appproach.)
The advantage is that maps are extensible. So, you can have middleware that e.g. checks authentication and authorization, adds keys to the map, that later code can check it directly. Namespacing guarantees nobody stomps on anyone else's feet. Spec/malli and friends tell you what to expect at those keys. You can sort of do the same thing in some other programming languages, but generally you're missing one of 1) typechecking 2) namespacing 3) convenience.
This is one of those self-inflicted Clojure problems. In Common Lisp you might use an alist or a plist for small things, but you'd definitely reach for CLOS classes for things that had relationships to other things and things that had greater complexity.
IIRC, the preference for complecting things via maps, and then beating back the hordes of problems with that via clojure.spec.alpha (alpha2?) is a Hickey preference. I don't recall exactly why.
No source to back this up, but my guess is that Clojure was driven by the need to interopt with Java so is to not get kicked out of production. This meant absorbing the Java object model. Shipping a language with both Java objects and CLOS and making them both play nice together sounds like a nightmare.
This comment helpfully explains many of the reasons Rich had for choosing immutable, persistent, generic data structures as the core information model in clojure (instead of concrete objects / classes): https://news.ycombinator.com/item?id=28041219
Not wanting to misquote the above / Rich himself I would TLDR it to:
- flexibility of data manipulation
- resilience in the face of a changing outside world
- ease of handling partial data or a changing subset of data as it flows through your program
Please note that no one (I hope) is saying that the above things are impossible or even necessarily difficult with static typing / OOP. However myself and other clojurists at least find the tradeoff of dynamic typing + generic maps in clojure to be a net positive especially when doing information heavy programming (e.g. most business applications)
Namedtuples FTW! A de-facto immutable dict with the keys listed right there in the definition to obviate all the usage head-scratching. Then, if you need more functionality (eg factory functions to fill in sensible defaults), you can just subclass it.
TBH I've never understood the attraction of the untyped dict beyond simple one-off hackups (and even there namedtuples are preferable), because like you say you typically have no idea what's supposed to be in there.
Question: 1. Can a GET request have a non-empty request body?
2. Assuming you don’t know the answer to that question, will the type system you use be able to tell you the answer to that question?
This is a pretty simple constraint one might want (a constraint that only certain requests have a body) but already a lot of static type systems (e.g. the C type system) cannot express and check it. If you can express that constraint, is it still easy to have a single function to inspect headers on any request? What about changing that constraint in the type system when you reread the spec? Is it easy?
The point isn’t that type systems are pointless but that they are different and one should focus on what the type system can do for you, and at what cost.
Any statically-typed language with generics can express that by parameterising the request type with the body type. A bodiless request is then just Request[Nothing] (or Request[Unit] if your type system doesn't have a bottom type). Accessing the headers just requires an interface which all static languages should be able to express.
To be fair, an incoming request is, almost by definition, dynamic. It makes sense to have that as a map, since the main sensible thing to do on receipt is validation/inspection.
Granted, you may have a framework do a fair bit of that. Depends how much you want between receipt of the request and code you directly control.
Usually the approach in a statically-typed language is to transform your dynamic request into something that you know through parsing instead of validation. Here's a great article about this: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va....
I do like static typing. But honestly, no other PL¹ (statically typed or otherwise) even comes close in terms of the ergonomics and joy of writing software. Nothing is quite enjoyable for me like Clojure. Haskell is great but hard, and I'm years away from claiming I achieved production-ready proficiency with it. I don't want to berate other languages, but I looked into OCaml, F#, Kotlin, Scala, Rust, and a few others. And none of them feel to me as enjoyable as Clojure.
After so many years of programming, I finally, truly feel like I love my job. Also, I never liked Python. Maybe just a little, in the beginning. Once I get to know it, I disliked it forever.
-------
¹ I mean languages used in the industry, not counting even more "esoteric" PLs
Walmart Labs was a step in this direction.. but we need some big companies to standardize around Clojure to jumpstart the ecosystem of knowledge, libraries, talent, etc. I’ve spoken to engineering hiring managers at fairly big companies and they’re not willing to shift to a niche language based only on technical merits but without a strong ecosystem.
If we don’t get some big companies to take on this roll the language is going nowhere.
I’m saying this because I’m a huge fan of Clojure (as a syntax and language, not crazy about the runtime characteristics) and I hope I get the opportunity to use it.
I started working professionally with Clojure earlier this year and this article rings true. I think the article leaves out a fourth downside to running on the JVM: cryptic stack traces. Clojure will often throw Java errors when you do something wrong in Clojure. It's a bit of a pain to reason about what part of your Clojure code this Java error relates to, especially when just starting out.
To be fair, this is not unique to Clojure. You need to deal with stack traces no matter what as long as you're using any programming language that targets the JVM (even statically type-checked languages like Scala). There are some great articles [1][2] that discuss various simple techniques helpful for debugging and dealing with stack traces.
I've never really had a problem with stack traces in Scala. Every once in a while you hit a cryptic one that's buried in Java library code, but for the most part they're runtime errors that are due to incompletely tested code or some kind of handled error with a very specific message.
I work at Ladder [0], and almost everything is done in Clojure/ClojureScript here. I had no previous experience in Clojure – Ladder ramps you if you haven't used it before. My interview was in Python. We're currently hiring senior engineers, no Clojure experience necessary [1].
I have had the pleasure of contributing to their code since we used their product at a previous company I worked at, and I must say I am sold on Clojure. Definitely a great language to have in your toolbox.
> We tried VisualVM but since Clojure memory consists mostly of primitives (Strings, Integers etc) it was very hard to understand which data of the application is being accumulated and why.
I was going to suggest this -- inside of VisualVM, you can right-click a process and then press "Start JFR"
Then wait a bit, right click it again, and select "Dump JFR"
What you get is a Flight Record dump that contains profiling information you can view that's more comprehensive than any language I've ever seen.
I used this for the first time the other day and felt like my life has been changed.
Specifically, if you want to see where the application is spending it's time and in what callstacks, you can use the CPU profiling and expand the threads -- they contain callstacks with timing
There's some screenshots in an issue I filed here showing this if anyone if curious what it looks like:
I tried to use Clojure but what put me of was that simple mistakes like missing argument or wrongly closed bracket didn't alert me until I tried running the program and then gave me just some java stack spat out by jvm running Clojure compiler on my program.
> didn't alert me until I tried running the program
That's because that's not how Clojure developers normally work. You don't do changes and then "run the program". You start your REPL and send expressions from your editor to the REPL after you've made a change you're not sure about. So you'd discover the missing argument when you call the function, directly after writing it.
Interesting. How exactly that looks? Do you have files opened in your editor, change them then go into previously opened repl, and just call the functions and the new version of those function runs?
Readit News
> To understand a program you must become both the machine and the program.
- Epigrams in Programming, Alan Perlis
Two of the big advantages of (gradually-) typed languages are communication (documentation) and robustness. These can be gained back with clojure spec and other fantastic libraries like schema and malli. What you get here goes way beyond what a strict, static type systems gets you, such as arbitrary predicate validation, freely composable schemas, automated instrumentation and property testing. You simply do not have that in a static world. These are old ideas and I think one of the most notable ones would be Eiffel with it's Design by Contract method, where you communicate pre-/post-conditions and invariants clearly. It speaks to the power of Clojure (and Lisp in general) that those are just libraries, not external tools or compiler extensions.
The current crop of statically typed languages (from the oldest ones, e.g. C#, to the more recent ones, e.g. Kotlin and Rust) is basically doing everything that dynamically typed languages used to have a monopoly on, but on top of that, they offer performance, automatic refactorings (pretty much impossible to achieve on dynamically typed languages without human supervision), fantastic IDE's and debuggability, stellar package management (still a nightmare in dynamic land), etc...
Only if you are skimping on tests. There's a tradeoff here - "dynamically typed" languages generally are way easier to write tests for. The expectation is that you will have plenty of them.
Given that most language's type systems are horrible (Java and C# included) I don't really think it's automatically a net gain. Haskell IS definitely a net gain, despite the friction. I'd argue that Rust is very positive too.
Performance is not dependent on the type system, it's more about language specification (some specs paint compilers into a corner) and compiler maturity. Heck, Javascript will smoke many statically typed languages and can approach even some C implementations(depending on the problem), due to the sheer amount of resources that got spent into JS VMs.
Some implementations will allow you to specify type hints which accomplish much of the same. Which is something you can do on Clojure by the way.
Automatic 'refactorings' is also something that's very language dependent. I'd argue that any Lisp-like language is way easier for machines to process than most "statically typed" languages. IDEs and debugability... have you ever used Common Lisp? I'll take a condition system over some IDE UI any day. Not to mention, there's less 'refactoring' needed.
Package management is completely unrelated to type systems.
Rust's robust package management has more to do with it being a modern implementation than with its type system. They have learned from other's mistakes.
Sure, in a _corporate_ setting, where you have little control over a project that spans hundreds of people, I think the trade-off is skewed towards the most strict implementation you can possibly think of. Not only type systems, but everything else, down to code standards (one of the reasons why I think Golang got popular).
In 2021, I would expect people to keep the distinction between languages and their implementations.
My conclusion is that it's a matter of personal preference honestly. Those are all really good languages. Personally I have more fun and enjoy using Clojure more. I would say I tend to find I'm more productive in it, but I believe that's more a result of me finding using it more enjoyable then anything else.
I don't pick Clojure for its dynamic typing, I pick it for other reasons. I've tried Haskell but it really doesn't seem to mesh with the way I tend to develop a program. But I would love to have more static languages with the pervasive immutability of Clojure.
...are we talking about the thing pioneered by Smalltalk's Refactoring Browser?
Deleted Comment
Deleted Comment
Dead Comment
Oh man. This is the fundamental disagreement. Sure, you can have a type system that is never wrong in its own little world. But, that's not the problem. A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all. It's like you got the wrong answer really, really right.
Yes this is the fundamental tradeoff. Specs et al are undoubtedly more flexible and expressive than static type systems, at the expense of some configurable error tolerance. I don't think one approach is generally better than the other, it's a question of tradeoffs between constraint complexity and confidence bounds.
It is a design decision to be able to build a clojure system interactively while it is running, so a runtime type checker is a way for the developer to give up the safety of type constraints for this purpose—by using the same facility we already need in the real world, a way to check the structure of data we can’t anticipate.
You just don't get an interpreter/compiler and have to sort everything else by yourself, no, there is a full stack experience and development environment.
Is this refinement types, which most static languages provide? https://en.wikipedia.org/wiki/Refinement_type
> freely composable schemas,
My understanding is that you can compose types (and objects) https://en.wikipedia.org/wiki/Object_composition
I'm assuming that types are isomorphic with schemas for the purposes of this discussion.
> automated instrumentation
I know that C# and F# support automated instrumentation/middleware.
> and property testing. You simply do not have that in a static world.
QuickCheck has entered the chat: https://en.wikipedia.org/wiki/QuickCheck
Well it does include that kind of behaviour but it's quite a bit more than just that. E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction. I'm not necesarily saying you should but just to give an illustrative example that there's less restrictions on your freedom to express what you need than in a static system.
>> types are isomorphic with schemas
I don't think that's a good way to think of this, you're imagining a rigid 1:1 tie of data and spec yet i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
With "freely composable" I mean that you can program with these schemas as they are just data structures and you only specify the things you want to specify. Both advantage and the disadvantage is that this is dynamic.
An even better example of this would be Excel, the horror stories are almost incredible.
So even if your environment is dynamic, you want clarity when you made a mistake. Handling errors gracefully and hiding them are very different things. The optimal in a dynamic world is to facilitate reasoning while not restricting expression.
Eg, spec can warn you when an argument doesn't make sense relative to the value of a second argument. Eg, with something like (modify-inventory {:shoes 2} :shoes -3) spec could pick up that you are about to subtract 3 from 2 and have negative shoes (impossible!) well before the function is called - so you can test elsewhere in the code using spec without having to call modify-inventory or implement specialist checking methods. And a library author can pass that information up the chain without clear English documentation and using only standard parts of the language.
You can't do that with defrecord, but it is effectively a form of documentation about how the arguments interact.
I like a REPL for testing things out, or for doing quick one-off computations, but that's it. I would never want to, say, redefine a function in memory "while the code is running". Not just because of ergonomics, but because if I decide to keep that change, I now have to track down the code I typed in and manually copy it back over into my source files (assuming I can still find it at all). And if I make a series of changes over a session, the environment potentially gets more and more diverged from what's in sourced if I forget to copy any changes over. So I'd often want to re-load from scratch anyway, at least before I commit.
Am I missing something? Am I misunderstanding what people mean when they talk about coding from a REPL?
You also can get an experience like PHP, where you just have to change and save some files, and your subsequent requests will use the new code. This is so much better than shutting down everything and restarting and is a large part of why CGI workflows dominated the web.
Common Lisp takes these experiences and dials them to 11, the whole language is built to reinforce the development style of dynamic changes, rather than an after-thought that requires a huge IDE+proprietary java agent. It's still best to use some sort of editor or IDE, and then you don't have any worry about source de-syncs -- frequently you'll make multiple changes across multiple files and then just reload the whole module and any files that changed with one function call, which you might bind to an editor shortcut, but crucially like debugging is not centrally a feature of the editor but the language; the language's plain REPL by itself is just a lot more supportive of interactive development than Python/JS/Ruby's. Clojure, and I personally think even Java with the appropriate tools, are between Python and CL for niceness of interactive development, but Clojure tends to be better than the Java IDE experience because of its other focus on immutability.
One of the best parts about lisp style repl development is that you end up doing TDD automatically. You just redefine a function until it does what you want from sample data you pass in - without changing files or remembering how your test framework works. You can save the output of some http call in a top level variable and iterate on the code to process it into something useful. The code you evaluate lives in the file that will eventually house it anyway so it’s pretty common to just eval the entire file instead of just one function.
Since you don’t ever shut the repl down, developing huge apps is also quite pleasant. You only reload the code that you’re changing - not the rest of the infra so things like “memoize” can work in local development. That’s why it’s a bit closer to your bash shell in other languages.
If you’ve never tried it, I highly recommend trying the Clojure reloaded workflow [1] to build a web app with a db connection. You can really get into a flow building stuff instead of waiting for your migrations to run on every test boot.
[1] https://cognitect.com/blog/2013/06/04/clojure-workflow-reloa...
That reads like what someone would think when they used the typesystem of Java 6 and now think that this is what "statically typed programming" means.
No, you can _not_ get back what types give you by any kind of spec - if anything you can get some of the benefits of types, but you also pay a price.
The thing is - dynamically typed languages don't really seem to evolve anymore. They add a bit of syntatic sugar here and there and sometimes add some cool feature, but mostly only features that already existed for a long time in other languages. At least that is what I have seen over the past couple of years, I would be happy to be proven wrong.
Looking at statical typesystems however, there is much more progress, simply because they are much more complex and not as optimized. From row-types over implicits and context-expressions towards fully fledged value-dependent typesystems, which have amazing features that start to slowly trickle down into mainstream languages like Typescript or Scala.
While both dynamically and statically typed languages have their pros and cons and it will stay like that forever, I expect that statically typed languages will proceed to become the bigger and bigger piece of the cake, simply because they have more potential for optimizations going forward.
I keep seeing lisp people bandy about all of this design by contract/arbitrary predicate validation stuff. Can you give an example of an instance in which static types + runtime checks don't completely subsume this?
My intuition is that almost all of these methods people are talking about would have to be enforced at run-time, in which case I don't see how it's providing anything fundamentally more than writing an assertion or a conditional.
Why static typing makes those things impossible?
I should have made clear that I'm emphasizing the advantages of being dynamic to describe and check the shape of your data to the degree of your choosing. Static typing is very powerful and useful, but writing dynamic code interactively is not just "woopdiedoo" is kind of the point I wanted to make without being overzealous/ignorant.
In an OOP language, types are hugely important, because the types let you know the object's ad-hoc API. OOP types are incredibly complicated.
In lisps, and Clojure in particular, your information model is scalars, lists, and maps. These are fully generic structures whose API is the standard Clojure lib. This means that its both far easier to keep the flow of data through your program in your head.
This gives you a 2x2 matrix to sort languages into, static vs dynamic, and OOP vs value based.
* OOP x static works thanks to awesome IDE tooling enabled by static typing
* value x static works due to powerful type systems
* value x dynamic works due to powerful generic APIs
* OOP x dynamic is a dumpster fire of trying to figure out what object you're dealing with at any given time (looking right at you Python and Ruby)
CLOS is "OOP x dynamic".
Common Lisp has arrays, structures and stack allocation.
IDE were invented from Smalltalk and Lisp development experience.
But then you get to Clojure proper, and you run into additional syntax that either convention or functions/macros that look like additional syntax.
Ok, granted, -> and ->> are easy to reason about (though they look like additional syntax).
But then there's entirely ungooglable ^ that I see in code from time to time. Or the convention (?) that call methods on Java code (?) with a `.-`
Or atoms defined with @ and dereferenced with *
Or the { :key value } structure
There's way more syntax (or things that can be perceived as syntax, especially to beginners) in Clojure than the articles pretend there is.
Finally,
is the same as and the `*` means nothing to the language – any character is valid on symbol names, the author just chose an asterisk.Most of what you believe to be syntax are convenience "reader macros" (https://clojure.org/reference/reader), and you can extend with your own. You can write the same code without any of it, but then you'll have more "redundant" parenthesis.
And yet, you need to know what all those ASCII symbols mean, where they are used, and they are indistinguishable from syntax.
Moreover, even Clojure documentation calls them syntax. A sibling comment provided a wonderful link: https://clojure.org/guides/weird_characters
Usually this is made worse by bespoke build tools and optimizations that make the system punishing to pick up.
"Normal language"?
You mean, whatever language is most popular at the company. What's "normal" at one would be completely alien at another. Even things like Java. If you don't have anything in the Java ecosystem, the oddball Java app will be alien and will likely get rewritten into something else.
The reason Clojure remains niche is that some people somehow think it's not a "normal" language, for whatever reason.
What clojure really needs is some kind of opinionated framework or starter template, something like create-react-app. That has all these things figured out so a beginner like me can start playing with actual clojure, which documents all the steps to setup the repl and editor and what not. The last time I asked for this I was told about lein templates, they help but there's no documentation to go with those.
There needs to be some push from the top level. create-react-app was produced by facebook. Elm reactor (which lets you just create a .elm file and play with elm) was created by Evan the language creator himself.
tldr: There's a huge barrier to start playing with clojure that needs to come down and the push needs happen from the top level.
I learned to code in Python. Loved it. Dynamically typed dicts up the wazoo!
Then I learned why I prefer actual types. Because then when I read code, I don't have to read the code that populates the dicts to understand what fields exist.
The advantage is that maps are extensible. So, you can have middleware that e.g. checks authentication and authorization, adds keys to the map, that later code can check it directly. Namespacing guarantees nobody stomps on anyone else's feet. Spec/malli and friends tell you what to expect at those keys. You can sort of do the same thing in some other programming languages, but generally you're missing one of 1) typechecking 2) namespacing 3) convenience.
[0]: spec-ulation keynote from a few years ago does a good job explaining the tradeoffs; https://www.youtube.com/watch?v=oyLBGkS5ICk
IIRC, the preference for complecting things via maps, and then beating back the hordes of problems with that via clojure.spec.alpha (alpha2?) is a Hickey preference. I don't recall exactly why.
Not wanting to misquote the above / Rich himself I would TLDR it to:
- flexibility of data manipulation
- resilience in the face of a changing outside world
- ease of handling partial data or a changing subset of data as it flows through your program
Please note that no one (I hope) is saying that the above things are impossible or even necessarily difficult with static typing / OOP. However myself and other clojurists at least find the tradeoff of dynamic typing + generic maps in clojure to be a net positive especially when doing information heavy programming (e.g. most business applications)
TBH I've never understood the attraction of the untyped dict beyond simple one-off hackups (and even there namedtuples are preferable), because like you say you typically have no idea what's supposed to be in there.
2. Assuming you don’t know the answer to that question, will the type system you use be able to tell you the answer to that question?
This is a pretty simple constraint one might want (a constraint that only certain requests have a body) but already a lot of static type systems (e.g. the C type system) cannot express and check it. If you can express that constraint, is it still easy to have a single function to inspect headers on any request? What about changing that constraint in the type system when you reread the spec? Is it easy?
The point isn’t that type systems are pointless but that they are different and one should focus on what the type system can do for you, and at what cost.
2. Sure. The request type has a body property.
Granted, you may have a framework do a fair bit of that. Depends how much you want between receipt of the request and code you directly control.
I do like static typing. But honestly, no other PL¹ (statically typed or otherwise) even comes close in terms of the ergonomics and joy of writing software. Nothing is quite enjoyable for me like Clojure. Haskell is great but hard, and I'm years away from claiming I achieved production-ready proficiency with it. I don't want to berate other languages, but I looked into OCaml, F#, Kotlin, Scala, Rust, and a few others. And none of them feel to me as enjoyable as Clojure. After so many years of programming, I finally, truly feel like I love my job. Also, I never liked Python. Maybe just a little, in the beginning. Once I get to know it, I disliked it forever.
-------
¹ I mean languages used in the industry, not counting even more "esoteric" PLs
Deleted Comment
If we don’t get some big companies to take on this roll the language is going nowhere.
I’m saying this because I’m a huge fan of Clojure (as a syntax and language, not crazy about the runtime characteristics) and I hope I get the opportunity to use it.
- Cisco - has built their entire integrated security platform on Clojure
- Walmart Labs and Sam's club - have some big projects in Clojure
- Apple - something related to the payment system
- Netflix and Amazon, afaik they use Clojure as well
even NASA uses Clojure.
I think the language "is going somewhere"...
[1] https://eli.thegreenplace.net/2017/notes-on-debugging-clojur...
[2] https://cognitect.com/blog/2017/6/5/repl-debugging-no-stackt...
[0] https://www.ladderlife.com/
[1] https://boards.greenhouse.io/ladder33/jobs/2436386
I have had the pleasure of contributing to their code since we used their product at a previous company I worked at, and I must say I am sold on Clojure. Definitely a great language to have in your toolbox.
You should try deeper profiling tools like JFR+JMC (http://jdk.java.net/jmc/8/) and MAT (https://www.eclipse.org/mat/).
Then wait a bit, right click it again, and select "Dump JFR"
What you get is a Flight Record dump that contains profiling information you can view that's more comprehensive than any language I've ever seen.
I used this for the first time the other day and felt like my life has been changed.
Specifically, if you want to see where the application is spending it's time and in what callstacks, you can use the CPU profiling and expand the threads -- they contain callstacks with timing
There's some screenshots in an issue I filed here showing this if anyone if curious what it looks like:
https://github.com/redhat-developer/vscode-java/issues/2049
Thanks Oracle.
It didn't feel like a first class experience.
That's because that's not how Clojure developers normally work. You don't do changes and then "run the program". You start your REPL and send expressions from your editor to the REPL after you've made a change you're not sure about. So you'd discover the missing argument when you call the function, directly after writing it.