huahaiy (u/huahaiy) - Readit News

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

yogthos · 6 months ago

The whole premise of static typing is that type errors can be caught at compile time. Java type system allows for kinds of errors to be caught at compile time that will not be caught by using protocols in Clojure. Yes, you can run the compiler at run time, but that's entirely besides the point here.

These libraries aren't using protocols for type safety though, they're using them as a performance optimization. That's certainly a perfectly fine reason to use protocols, and I agree that it's a completely legitimate use case. It's the whole right tool for the job thing. If you're writing something where performance is the top concern, then that's what protocols are for.

I very much agree with you that that there are people who focus on low level code, and those who focus on application level code. The style of coding will be different depending on the type of problem you're solving. You're right that I failed to qualify my original statement regarding protocols being an anti-pattern.

You're absolutely correct that we should take a pragmatic approach towards using language features. Hence why the context of whether protocols are the right tool to each for lies in the type of code you're writing. And of course, some people find it easier to have more structure to help with their reasoning. Although, I'd argue tools like Malli work better there.

To sum up, I'm not arguing against protocols being useful or that there's no place for them. We started this discussion talking about whether protocols provide equivalent guarantees to Java's type system. I disagree regarding that. However, I also don't think that this is a real problem. Otherwise, use of something like core.typed would've become prevalent by now.

huahaiy · 6 months ago

I wrote my library this way for both performance AND type checking reasons. You cannot tell me the reasons why I wrote my code my way. That is just absurd.

In any case, it is possible to write Clojure in a way that is type checked at compile time. And it is an acceptable way to write Clojure. I just want to clear the air.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

lkitching · 6 months ago

Of course Clojure has to ultimately be compiled into a native format for the host platform, bytecode in the case of the JVM implementation, but that doesn't require type checking in the same way Java does.

Clojure functions are compiled into implementations of clojure.lang.IFn - you can see from https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.h... that this interface simply has a number of overloads of an invoke method taking variable numbers of Object parameters. Since all values can be converted to Object, either directly for reference types or via a boxing conversion, no type checking is required to dispatch a call. With a form like

  (some-fn 1, "abc", (Object.))

the some-fn symbol is resolved in the current context (to a Var for functions defined with defn), the result is cast (not checked!) to an instance of IFn and the call to the method with required arity is bound. This can go wrong in multiple ways: the some-fn symbol cannot be resolved, the bound object doesn't implement IFn, the bound IFn doesn't support the number of supplied arguments, the arguments are not of the expected type. Clojure doesn't check any of these, whereas the corresponding Java code would.

Protocol methods just get compiled into an implementation of IFn which searches for the implementation to dispatch to based on the runtime type of the first argument, so it doesn't introduce static type checking in any way.

huahaiy · 6 months ago

But if you add type hint in the signature, it does check the type. Basically, if you specify the type, it will check type. Just like any language that is not automatically inferring types, e.g. Java. So it is the same as Java.

You guys make it out like Clojure is doing something extra to hide Java types, but it doesn’t. What Clojure does is really minimal on top of Java. It barely hides anything.

If you give it type, it will check type. If you don’t give a type, it falls back to a default type, Object, which IS a TYPE. The fact that Clojure compiler cannot deal with GraalVM SVM Pointer type tells you that it’s checking type, because Pointer is not an Object! I found this out the hard way: https://yyhh.org/blog/2021/02/writing-c-code-in-javaclojure-...

“One limitation that one needs to be aware of when writing native image related Clojure code, is that most things in the GraalVM SDK inherit from org.graalvm.word.WordBase, not from java.lang.Object, which breaks the hidden assumption of a lot of Clojure constructs.”

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

yogthos · 6 months ago

Even if you did, that wouldn't solve the problem because many checks are still done at runtime. Also, if you started doing that then you might as well just write Java at that point. I've worked on code bases structured in this way and it's absolutely terrible to work with. For one, protocols completely break any sort of REPL driven development.

Most libraries are absolutely not written in this way either. Please point me to a single library that's actually written in the style you describe. The use of protocols in actual popular libraries like Ring tends to be minimal.

The reality is that dynamic typing has never been a real problem in Clojure. I've worked with the language almost exclusively for over a decade now, and I maintain a number of popular libraries, like Selmer, with millions of users.

huahaiy · 6 months ago

That has nothing to do with types. You are now talking about when a compiler is run.

Clojure gives you the option to run the compiler at runtime, so that's what people normally do. However, you can also run the compiler at compile time. Right?

For people who want type checking, they can opt to write Clojure in this defprotocol everywhere style, and turn on AOT. Then they basically get the same thing as what Java gives them.

As to example of libraries that are defprotocols everywhere, you should look at any of the low level performance minded libraries in the Clojure ecosystem, they are either written in this defprotocols everywhere style, such as nippy, neanderthal, dtype-next, and so on, or mostly in Java, such as http-kit, fast-edn, etc. I noticed this phenomenon, because my own libraries, editscript and datalevin, are written in this way. I take comfort that my fellow performance minded library authors are doing the same. Finally, isn't Clojurescript entirely driven by protocols?

So really, there are two kind of Clojure programmers. One type writes application code or high level libraries, and they write normal Clojure code all the time. However, there are also those who write primarily low level library code, which are used by the first camp, and their code is full of defprotocol and deftypes. So defprotocol everywhere is not anti-pattern. It's anti-pattern only in the mind of the first camp of programmers, and that's a narrow minded way of looking at things. Even the first version of Clojure Programming book by the core team members, are written in a way that's full of defrecord. Remember?

This "everything is a map" orthodoxy is turning people away from Clojure. Just let people write the code that suits their own needs. We can use more people who are pragmatic instead of dogmatic in the Clojure world. If you trust Rich Hickey's judgment, then you should trust him put in the features Clojure has for good reasons. Macros and protocols are part of the Clojure language, and you should be using them when the use case calls for them. Stop the "anti-pattern" nonsense.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

lkitching · 6 months ago

Protocols do not work like Java interfaces or classes. Their methods are compiled into regular functions which lookup the implementation to use at runtime based on the runtime type of the receiver. Compilation will check for the named function but doesn't do any further checking. Given the following protocol and implementation:

  (defprotocol P
    (method [this ^Integer i]))

  (extend-protocol P
    String
    (method [s i] (.substring s i)))

both (method "test" "call") and (method 1 2) will be accepted by the compilation phase but will fail at runtime.

Of course there's no requirement for Clojure code to be AOT compiled anyway so in that case any name errors will still only be caught at runtime when the compilation happens.

Type hinted bindings are only converted into a cast and are not checked at compilation time either e.g.

  (defn hinted [^String s] (.length s))
  (hinted 3)

will be accepted but fail at runtime.

deftype is only used for Java interop an is also not a form of type checking. The methods will be compiled into Java classes and interfaces, but the implementations defer to regular Clojure functions which are not type checked. You can only make use of the type information by referencing the compiled class files in Java or another statically typed language, using them from Clojure will not perform type checking.

huahaiy · 6 months ago

deftype IS a Java class, it's not compiled into something else. What is a Clojure function? A Clojure function is a Java class. Clojure is a compiled language, so it does check types, just like Java check types.

So if you use defprotocol and deftype for every domain objects in your code, your code won't compile if there's a type error. Try it.

BTW, that's the way many Clojure libraries are implemented. These libraries rely on dispatch on type to work, so they are taking advantage of the type checking.

Of course, you will say, "oh, clojure is not normally AOT, so it's not dong the checks.", but that's another issue. The issue at hand is this: can you write Clojure such that types are checked at compile time. The answer is YES.

The compiler may run only when you run the program, that's a different issue. You are confusing these two issues.

If you want a separate compile stage, then basically you are already excluding runtime compilation, i.e. you are arguing against runtime compilation. So it's not really about typing, but about how you want to run the program. Isn't it? You want AOT for everything, you don't want runtime compilation. That's it. It has nothing to do with types.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

yogthos · 6 months ago

That's not quite true though. Java tracks types in the signatures of the functions, defprotocol does not. If I make a protocol and then pass a wrong type as a parameter to it then I'll get a runtime error. It's not going to tell me that I passed in a wrong type at compile time.

I find using defprotocol in Clojure tends to be an antipattern because it just makes code harder to read by introducing indirection. The libraries using defprotocol use it to solve a specific problem of creating a contract for how the API looks.

huahaiy · 6 months ago

Then you are not writing in defprotocol everywhere style. The keyword is everywhere. All the domain objects are deftype or defrecord. Try that. It is the same as Java, basically.

It is not an anti pattern, it is the way most low level libraries and clojure itself are written.

Clojure is a tool, not a cult. This core team worship is turning people away. The core team made plenty of mistakes, and got called out, rightfully.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

lkitching · 6 months ago

Neither defprotocol nor deftype introduce static typing into Clojure. Errors in their usage are not checked statically and are only discovered at runtime.

huahaiy · 6 months ago

No, defprotocol and deftype have the same properties as Java interface and Java classes, and the types are checked at compile time. This is static typing. Period. Clojure is a compiled language, it does check typing during compilation.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

yogthos · 6 months ago

I'd argue defprotocol, deftype, and defrecord provide much weaker guarantees than a type system. Dtuff like defprotocol tends to serve a similar use case to using an interface in Java. You specify the signatures for the functions, and then a concrete implementation can be provided using a library. Ring servers are a good example of this where you can easily plug different server implementations by just swapping a library.

There is Typed Clojure https://typedclojure.org/ for people who want actual static typing, but the fact that it never caught on suggests this wasn't a real pain point for most people using Clojure.

As other people mentioned, immutability tend to be a more important feature than static types because it makes it easy to write code that's referentially transparent. You can structure your whole application as a series of small components that can be reasoned about in isolation.

huahaiy · 6 months ago

Nobody would argue that Java is not statically typed. That's my point. Clojure offers the same as what Java offers.

If you write code in a defprotocol everywhere style, as many Clojure libraries do, your code won't compile if you got the types wrong. The same as Java. How's that not static typing? Which part of that is weaker?

So what exactly this "Clojure dynamic typing" nonsense is about, I fail to see.

Automatically inferred type system is not the same thing as static typing. Typed Clojure is the former. Typed Clojure did not catch on, but static typing style of Clojure did, as many Clojure libraries do exactly that: internally, defprotocal everywhere, externally, some Clojure functions to give the illusion of normal Clojure code. BTW, that's the style how Clojure itself is written in as well.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

rednafi · 6 months ago

It’s probably geared toward someone like me. JVM and parens are one angle.

Another is that I get paid to work with distributed systems and databases, and Clojure isn’t even part of the discussion in that sphere. Go, Rust, and the usual Python and Node dominate there, so it’s hard for me to care.

However, I dislike language monoculture and am curious about why people like the things I might not care about. This blog is for those.

huahaiy · 6 months ago

I thought distributed system is exactly one of the things that got Clojure going to begin with. Remember Storm?

As to databases, there are a fountain of databases emerging from the Clojure ecosystem: Datomic, Datascript, Datalevin, XTDB, and friends aside, there is Rama, which is a distributed database (or to replace database?)

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

jayceedenton · 6 months ago

10+ years ago Clojure had a fantastic introductory experience. lein new and away you go. lein was so good and effective, for both tiny hello world projects and real production apps.

The experience has gotten worse and worse now for a decade. The core team have continued to take things in a worse direction (supported by a small group of fanboys) and most newcomers are now completely baffled by the tooling.

huahaiy · 6 months ago

You can still do lein new and away you go, today. That still works.

Many people still use lein for new projects, especially for larger ones. For small on-off thing, clojure command line is more convenient. So it is a good thing to have more choices.

huahaiy commented on Why Clojure? gaiwan.co/blog/why-clojur... · Posted by u/jgrodziski

gavmor · 6 months ago

I played with Clojure just a bit in 2014 because I wanted to write GUIs in Om, and this gave me a seriously warped habit of calling React.el('div',...) for a while. Sorry not sorry.

I'm used to using TDD for fast feedback as I'm molding my code. Do you miss unit testing? Or, do you find that the REPL in no way obviates unit testing?

And, do you miss static typing?

huahaiy · 6 months ago

REPL code is copy/pasted straight into tests. So really, REPL is for helping write tests.

BTW, when Clojurians talk about REPL, it's not about that separate window where you type and run the code as in other language such as python. They are talking about an invisible REPL running behind the scene, to which they send code within their editors, and the results show up in the editors too.

There's no need to "miss static typing" in Clojure. If I need static typing, I just write deprotocol and deftype in Clojure, as many Clojure libraries do.