> Just to put this paper in an appropriate historical context, I wrote it back when I was trying to convince NASA that flying Lisp on a spacecraft would not be a completely insane thing to do. The Powers That Be had already mostly accepted that it would be OK to fly Java, but they thought Lisp was too big and slow. This study and paper were designed specifically to address those concerns.
Thanks dang - i appreciate when you post these, particularly on perennial topics: it's interesting to look back and see how the crowd's wisdom has(n't) changed.
> the directions said to code as you would professionally, and that means I should comment my work.
Were this a "professional" component/service, in perhaps the majority of situations today, he'd probably separate his docs into two kinds of documentation: the committed interface/API what, and the internal maintenance how.
The docstring for `print-translations`, for example, conflate these two kinds.
Incidentally, by 2000, Java had emphasized reuse API documentation for a few years, with how it presented Javadoc for both the core Java API and layered development, from the start.
By contrast, Lisp environments that exposed docstrings in IDE hypertext, like Emacs did, did some doc things better than Java for decades before, and are still really well-suited to army-of-one genetically-enhanced supersoldier Lisp hackers.
Enterprise development, SaaSes, microservices, large systems with multiple teams, etc., were arguably better off heading in the more software engineering direction that Java was going.
(Which is not to say that Lisps can't do the what&how docs separation. I've done a few embedded API docs things for Scheme and Racket that initially were inspired by Javadoc, Perl, and some of the less-commonly-used embedded docs setups for other languages. And have worked on a couple large Lisp systems that arguably wouldn't have been possible in a language without some of the Lisp family's other still-unusual strengths, nor without the highly-skilled developers that Lisps seem to attract disproportionately.)
Hunchentoot attempts to do the "what&how docs separation", but when I last used it, I had to look at the implementation to understand what my code was expected to do. At least Emacs and SLIME made it easy to pull up the source code.
Lisp is a wonderful language (Common Lisp in particular, but I am dabbling in Scheme now too), and absolutely fantastic for web development - seriously people should give it a chance in that space, I would totally develop any SPA in Lisp[1].
QuickLisp is simply a joy to use, and the language is very portable with a fantastic specification and many top notch compilers.
IMO, what holds it back is a lack of a top notch IDE. It took me a year of love & hate with Emacs, I can now make pretty good use of the amazing software that is Emacs (where everything can be changed), but honestly at times I want a simple point & click free IDE which is polished and does most things well and out of the box. Portacle is a step in the right direction, but the Emacs dependency IMO does make the language a bit more niche and lose out to other languages for beginners. If some big corporation can mix sublime with slime with paredit with some of the customize ability of Emacs, then Common Lisp could explode in popularity.
Don’t get me wrong - I LOVE Emacs and everything about Emacs & CL combined, but it took me a year to get to this point.
The emacs bias is even part of Lisp advocates mental models, so it's no surprise that a lot of would-be Lispers give up at the thought of having to use it. But there are alternatives, and it might be fruitful for advocates to routinely point them out. vim (my preference) has two slime plugins that work really well, Atom has https://atom.io/packages/slima and as you point out there's lots of Emacs-variants. There's also lem and Second-Climacs might one day be good. And of course there's the two proprietary IDEs with LispWorks and Allegro that both provide trial versions to see if you like them before shelling out money.
Apart from editors, more editor-independent tools would be nice. McCLIM's Clouseau is a pretty cool graphical inspector/variable watcher.
IMO what holds it back is lack of a big company using and promoting it along with tooling (which might be in the form of alternate IDEs or good plugins for popular editors like VS Code or IntelliJ). Clojure has shown what's possible without such, most use emacs but there are popular alternatives too, and for a time it was even as popular as Go, but Go has BigCo backing, and we can see how much further that brought it despite its many shortcomings.
BTW, there's a small small-world coincidence, between a desire for an Emacs-based super IDE, and the original article...
The desire for an Emacs-based out-of-box power-user IDE was how we got Lucid Emacs, aka XEmacs (which was one of the two big forks of GNU Emacs with GUI innovations that GNU took a while to catch up with)...
...from JWZ, et al.'s work on the Lucid Energize IDE for C++...
...and JWZ worked for Norvig early on...
...and Norvig was at Harlequin...
...and Harlequin acquired at least some of Lucid's IP.
> IMO, what holds it back is a lack of a top notch IDE
I hear good things about Lispworks[1][2], but personally I'm put off by non-free development tools. Especially ones with a 3-4 digit price tag. However, their business model evidently works for them.
He wrote this around the same time that he kindly reviewed and gave me comments on the first edition of my Java AI book. A year later we had lunch: I was using Common Lisp and Java for my work and I think that Peter was mostly using Python by then.
Which programming language you choose really doesn't matter very much. Since 1982 Common Lisp has been my favorite language, but I have recently learned to also love Swift.
The year after he wrote this, Norvig joined Google and decided that Python was a good enough alternative. He posted a web page about it, but I can't find it rn. However, here's a comment of his on HN about it:
How many of those 1500 QuickLisp libraries work together without extra work?
I seriously tried to use Common Lisp for numerical programming 10 years ago. It was faster and would have been better platform for numerical programming than Python, R or Julia. Just a great language. (Ross Ihaka had plans to port R into Common Lisp to make R fast at one point)
The real problem was the lack of convergence of data structures and conventions above the language standard.
Python has pandas tables and numpy arrays. R has Data frame. Every numerical library for CL had their own conventions, and ways to implement the basic data structure, Some used lists, some just arrays, some had list of vectors, classes, structs, property lists.
The impedance mismatch was not worth it. Either implement numpy equivalent on your own or half of your code is data massaging data between libraries.
--
Perlis epigram: "9. It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."
The philosophy of Lisp encourages writing your own libraries because it's so easy to do. You're expected to know how to program if you use Lisp; not to depend on tiny prewritten functions in the npm universe that may or may not solve your problem, that may or may not be written in a secure or efficient fashion, and that possibly pull in a million transitive dependencies you don't need.
That said, Common Lisp itself includes many of the more useful but difficult-to-write features like hash tables, sort/merge, bignums, complex numbers, etc.
The most-used stuff that's not in Common Lisp itself like multithreading and sockets are well-covered by external libraries. The lesser-needed stuff is also quite well covered by the quicklisp library infrastructure [0] as well.
So the statement "libraries...do not exist in the Lisp ecosystem" is plainly false and it marks the speaker as someone not very familiar with Lisp.
> The philosophy of Lisp encourages writing your own libraries because it's so easy to do.
Having professionally worked in CL and Java, this is true and completely misses the point. Of course you can write your own libraries, the question isn’t whether you can but whether or not that’s a wise usage of your time and energy. Java gives me a ton of fantastic libraries out of the box that let me focus on my domain, which is a good thing!
> You're expected to know how to program if you use Lisp; not to depend on tiny prewritten functions in the npm universe that may or may not solve your problem, that may or may not be written in a secure or efficient fashion, and that possibly pull in a million transitive dependencies you don't need.
Sneering at people who use different languages is both a bad look, and a bad way to convince people that you’re right. I’d recommend staying way clear of any suggestion that your peers don’t know how to program.
> The most-used stuff that's not in Common Lisp itself like multithreading and sockets are well-covered by external libraries. The lesser-needed stuff is also quite well covered by the quicklisp library infrastructure [0] as well.
This is a pretty drastic understatement of how far behind Java CL is. Forget multithreading, think more about metrics, logs, configuration, A/B testing, retries/bulkheads/circuit-breakers, feature flags, and secrets management. All of this stuff I get for free out of Java libraries, often in automatically self configuring ways.
Configuring a Kafka reader in my app and seeing that it automatically discovered Micrometer on my class path and started emitting metrics is a lovely experience. Switching metrics servers from InfluxDB to Datadog with a single code change was damn near nirvana.
If CL wants to be a compelling alternative to Java for industrial use, it’s got to catch up on all this stuff.
> So the statement "libraries...do not exist in the Lisp ecosystem" is plainly false and it marks the speaker as someone not very familiar with Lisp.
Sorry to be so blunt but this is ridiculous. You are mentioning hash tables and multithreading to prove your point. I am pretty sure this is not what parent meant when they wrote "The ecosystem is just not there". And what does "writing your own libraries because it's so easy to do" even mean? You still have to do it and it takes a lot of time and money. A library that talks a specific protocol to a medical devices. A library that writes some obscure data format used by some government somewhere on earth. This is the real world.
> The philosophy of Lisp encourages writing your own libraries because it's so easy to do. You're expected to know how to program if you use Lisp; not to depend on tiny prewritten functions in the npm universe that may or may not solve your problem, that may or may not be written in a secure or efficient fashion, and that possibly pull in a million transitive dependencies you don't need.
Some people love to point out how stupid JS programmers are because of the left-pad incident.
The reality is that, in almost every domain, there are many things which are either difficult and/or time-consuming to implement, or common across several projects and tasks. Do you really want to waste your own time and the time of your teammates reimplementing these things for every project if you don't need to?
Even if the underlying implementation is "easy", having a handful of well-tested libraries with thoughtfully-designed APIs is important for productivity in many (most?) programming tasks. Yes, there might always be a specific case when you can't use the popular library for X, but that's the exception and not the norm.
"It's easy to write your own libraries" is true in any professional-grade language. Arguably it's one of the things that makes a professional-grade language professional-grade. It's not a justification for having a bad library ecosystem, it's a rationalization.
As a side note, I think the Lisp library ecosystem in 2021 is pretty good, considering how unpopular it is in industry.
> The philosophy of Lisp encourages writing your own libraries because it's so easy to do
That statement could be said of any language the programmer is comfortable with. It's no more a LISP philosophy than a C++ one. The difference is that way more people are working with C++ (or Java) thus opensourcing their libraries for others to be used.
> You're expected to know how to program if you use Lisp
I am pretty sure you are expected to know how to program if you use any programming language. And writing everything from scratch, instead of taking advantage of the work done by thousands of other skilled developers, is just silly. With that attitude, no wonders the world runs on C/C++ and not Lisp.
Sure, libraries exist in Lisp. The claim that they don't was a fairly ridiculous statement. But...
Do libraries exist in Lisp on the same scale as they exist in Java? Especially, do they exist with the same level of quality, professionalism, and maintenance as they exist in Java? I'm guessing that the answer is no.
But you just touched on the reason programmers don't like Lisp: they prefer a language that has a lot of little bits of available functionality that they can mix and match, without having to do any real programming. Programming, as we knew it, is a dead art, modern developers only want to reuse some library that was created by someone else. Languages like Python and Java are bad to program, but ideal for this kind of lego-based development.
So true, but this class of comment always amuses me as in the 1980s one of the (many) criticisms leveled at Common Lisp was that its library support was too big.
Lisp isn't very popular because, at least in my opinion, it requires a relatively high level of maturity and experience to be used effectively on nontrivial projects.
Every single project in Lisp (Common Lisp, Clojure) that I have observed ended in a disaster mostly because developers could not figure out what the strengths of Lisp are to realize the benefits and then killed themselves with too much freedom.
Absent large enough number of success stories companies will default on Java because it is good enough language to be able to effectively realize typical enterprise application and dumb enough to be used by a beginner dev, even at the cost of boilerplate.
At least that's the official story. It is really difficult to explain to business that you could do whatever they want with 1/5th or 1/50th of the amount of code if you just used a good language for it.
25 years of programming experience here. I have written my own Lisp runtimes, micro kernels, C compilers, high performance web servers, a million selling game written in C, high performance pairing/rostering optimisers for the airline industry etc. I am a language geek and have programmed in pretty much everything I could get my hands on. And are currently maintaining a high performance compiler for a Haskell style language, and have fun learning how to prove code correct using Coq/F*/Idris/Lean. Is that mature enough for you? I would have zero problem writing applications in Lisp. But I choose not to. Why? Because I am more productive in other languages. That might not be the case for you. However your N=1 experiment doesn’t in any way generalise to N=everybody else.
I think it is a matter of education. Software developers learn to program in traditional languages, and Lisp is just an add on. So they really don't know how to use Lisp well. Languages like Python and Java benefit from all the formal education programmers have received in the concepts they use.
> it requires a relatively high level of maturity and experience to be used effectively on nontrivial projects.
Other languages scale smoothly from beginner to expert. So I guess that your assertion shows that Lisp is a less useful language?
Experienced programmers using non-Lisp languages are successful every day. Your OS/mobile phone/browser/… is the result of their work. So I guess other languages are better for experienced developers as well?
In other words, the real world evidence seems to suggest that Lisp is not good for beginners and not good for experienced developers.
There was an article in the 2000s from a dude looking at a graph library, java had 3 major players, non of them did what he needed (generics issues or similar). He ended up rolling his own.
I think the whole issue of having a great programming language is a mute point in the current software industry, because average programmers don't know how to use it. That's why the winners have always be languages that appeal to the minimum common denominator, just the bare enough for people to write programers in the simple way they know. Lisp provide a lot of flexibility and power, but if programmers don't know how to use it, it is instead viewed as a liability.
Tiny niggle about a well-known phrase and a word that is not particularly well-known outside of the phrase anymore: the word is moot. (A moot used to be a meeting or assembly of people who would discuss and decide things, often Very Important Things™. Nowadays, moot is mostly an adjective referring to things that might be debatable, but the only benefit to debating them would be the debate itself.)
One can still say "to moot an issue for discussion (bring up an issue)" but what also fascinates me is the other meaning for mute involving bird evacuation. I have to wonder what happened in Latin?
Readit News
Lisp as an alternative to Java (2000) - https://news.ycombinator.com/item?id=26720403 - April 2021 (94 comments)
Lisp as an Alternative to Java – Faster, More Productive - https://news.ycombinator.com/item?id=21899854 - Dec 2019 (1 comment)
Lisp as an Alternative to Java (1999) - https://news.ycombinator.com/item?id=12197131 - July 2016 (103 comments)
Lisp as an Alternative to Java (2000) [pdf] - https://news.ycombinator.com/item?id=9045574 - Feb 2015 (21 comments)
Lisp as an alternative to Java (2000) [pdf] - https://news.ycombinator.com/item?id=8446368 - Oct 2014 (55 comments)
Lisp as an Alternative to Java - https://news.ycombinator.com/item?id=3447101 - Jan 2012 (37 comments)
Lisp as an Alternative to Java [PDF] - https://news.ycombinator.com/item?id=61320 - Sept 2007 (9 comments)
> Just to put this paper in an appropriate historical context, I wrote it back when I was trying to convince NASA that flying Lisp on a spacecraft would not be a completely insane thing to do. The Powers That Be had already mostly accepted that it would be OK to fly Java, but they thought Lisp was too big and slow. This study and paper were designed specifically to address those concerns.
https://news.ycombinator.com/item?id=8446685
Deleted Comment
Were this a "professional" component/service, in perhaps the majority of situations today, he'd probably separate his docs into two kinds of documentation: the committed interface/API what, and the internal maintenance how.
The docstring for `print-translations`, for example, conflate these two kinds.
Incidentally, by 2000, Java had emphasized reuse API documentation for a few years, with how it presented Javadoc for both the core Java API and layered development, from the start.
By contrast, Lisp environments that exposed docstrings in IDE hypertext, like Emacs did, did some doc things better than Java for decades before, and are still really well-suited to army-of-one genetically-enhanced supersoldier Lisp hackers.
Enterprise development, SaaSes, microservices, large systems with multiple teams, etc., were arguably better off heading in the more software engineering direction that Java was going.
(Which is not to say that Lisps can't do the what&how docs separation. I've done a few embedded API docs things for Scheme and Racket that initially were inspired by Javadoc, Perl, and some of the less-commonly-used embedded docs setups for other languages. And have worked on a couple large Lisp systems that arguably wouldn't have been possible in a language without some of the Lisp family's other still-unusual strengths, nor without the highly-skilled developers that Lisps seem to attract disproportionately.)
QuickLisp is simply a joy to use, and the language is very portable with a fantastic specification and many top notch compilers.
IMO, what holds it back is a lack of a top notch IDE. It took me a year of love & hate with Emacs, I can now make pretty good use of the amazing software that is Emacs (where everything can be changed), but honestly at times I want a simple point & click free IDE which is polished and does most things well and out of the box. Portacle is a step in the right direction, but the Emacs dependency IMO does make the language a bit more niche and lose out to other languages for beginners. If some big corporation can mix sublime with slime with paredit with some of the customize ability of Emacs, then Common Lisp could explode in popularity.
Don’t get me wrong - I LOVE Emacs and everything about Emacs & CL combined, but it took me a year to get to this point.
[1] For those interested, I wrote this article covering the basics: https://ashok-khanna.medium.com/lisp-the-web-4c00c88d11f9
Apart from editors, more editor-independent tools would be nice. McCLIM's Clouseau is a pretty cool graphical inspector/variable watcher.
IMO what holds it back is lack of a big company using and promoting it along with tooling (which might be in the form of alternate IDEs or good plugins for popular editors like VS Code or IntelliJ). Clojure has shown what's possible without such, most use emacs but there are popular alternatives too, and for a time it was even as popular as Go, but Go has BigCo backing, and we can see how much further that brought it despite its many shortcomings.
The desire for an Emacs-based out-of-box power-user IDE was how we got Lucid Emacs, aka XEmacs (which was one of the two big forks of GNU Emacs with GUI innovations that GNU took a while to catch up with)...
...from JWZ, et al.'s work on the Lucid Energize IDE for C++...
...and JWZ worked for Norvig early on...
...and Norvig was at Harlequin...
...and Harlequin acquired at least some of Lucid's IP.
I hear good things about Lispworks[1][2], but personally I'm put off by non-free development tools. Especially ones with a 3-4 digit price tag. However, their business model evidently works for them.
[1] http://www.lispworks.com/products/lispworks.html
[2] https://lisp-journey.gitlab.io/blog/discovering-the-lispwork...
Which programming language you choose really doesn't matter very much. Since 1982 Common Lisp has been my favorite language, but I have recently learned to also love Swift.
All the libraries - both open source and commercial that do a million things - do not exist in the Lisp ecosystem.
Even in 2000
https://news.ycombinator.com/item?id=1803815
This essay starts with the context of his choosing Python to express what was formerly pseudocode.
https://news.ycombinator.com/item?id=1803627
The bigger issue though is wasting time wrapping Java libraries and most comments I see about Clojure interop talk as if there is no cost to it.
Clojure has good foundations but the ecosystem never really developed.
https://abcl.org/trac/
I seriously tried to use Common Lisp for numerical programming 10 years ago. It was faster and would have been better platform for numerical programming than Python, R or Julia. Just a great language. (Ross Ihaka had plans to port R into Common Lisp to make R fast at one point)
The real problem was the lack of convergence of data structures and conventions above the language standard.
Python has pandas tables and numpy arrays. R has Data frame. Every numerical library for CL had their own conventions, and ways to implement the basic data structure, Some used lists, some just arrays, some had list of vectors, classes, structs, property lists.
The impedance mismatch was not worth it. Either implement numpy equivalent on your own or half of your code is data massaging data between libraries.
--
Perlis epigram: "9. It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."
That said, Common Lisp itself includes many of the more useful but difficult-to-write features like hash tables, sort/merge, bignums, complex numbers, etc.
The most-used stuff that's not in Common Lisp itself like multithreading and sockets are well-covered by external libraries. The lesser-needed stuff is also quite well covered by the quicklisp library infrastructure [0] as well.
So the statement "libraries...do not exist in the Lisp ecosystem" is plainly false and it marks the speaker as someone not very familiar with Lisp.
[0] https://www.quicklisp.org/beta/releases.html
Having professionally worked in CL and Java, this is true and completely misses the point. Of course you can write your own libraries, the question isn’t whether you can but whether or not that’s a wise usage of your time and energy. Java gives me a ton of fantastic libraries out of the box that let me focus on my domain, which is a good thing!
> You're expected to know how to program if you use Lisp; not to depend on tiny prewritten functions in the npm universe that may or may not solve your problem, that may or may not be written in a secure or efficient fashion, and that possibly pull in a million transitive dependencies you don't need.
Sneering at people who use different languages is both a bad look, and a bad way to convince people that you’re right. I’d recommend staying way clear of any suggestion that your peers don’t know how to program.
> The most-used stuff that's not in Common Lisp itself like multithreading and sockets are well-covered by external libraries. The lesser-needed stuff is also quite well covered by the quicklisp library infrastructure [0] as well.
This is a pretty drastic understatement of how far behind Java CL is. Forget multithreading, think more about metrics, logs, configuration, A/B testing, retries/bulkheads/circuit-breakers, feature flags, and secrets management. All of this stuff I get for free out of Java libraries, often in automatically self configuring ways.
Configuring a Kafka reader in my app and seeing that it automatically discovered Micrometer on my class path and started emitting metrics is a lovely experience. Switching metrics servers from InfluxDB to Datadog with a single code change was damn near nirvana.
If CL wants to be a compelling alternative to Java for industrial use, it’s got to catch up on all this stuff.
Sorry to be so blunt but this is ridiculous. You are mentioning hash tables and multithreading to prove your point. I am pretty sure this is not what parent meant when they wrote "The ecosystem is just not there". And what does "writing your own libraries because it's so easy to do" even mean? You still have to do it and it takes a lot of time and money. A library that talks a specific protocol to a medical devices. A library that writes some obscure data format used by some government somewhere on earth. This is the real world.
Some people love to point out how stupid JS programmers are because of the left-pad incident.
The reality is that, in almost every domain, there are many things which are either difficult and/or time-consuming to implement, or common across several projects and tasks. Do you really want to waste your own time and the time of your teammates reimplementing these things for every project if you don't need to?
Even if the underlying implementation is "easy", having a handful of well-tested libraries with thoughtfully-designed APIs is important for productivity in many (most?) programming tasks. Yes, there might always be a specific case when you can't use the popular library for X, but that's the exception and not the norm.
"It's easy to write your own libraries" is true in any professional-grade language. Arguably it's one of the things that makes a professional-grade language professional-grade. It's not a justification for having a bad library ecosystem, it's a rationalization.
As a side note, I think the Lisp library ecosystem in 2021 is pretty good, considering how unpopular it is in industry.
That statement could be said of any language the programmer is comfortable with. It's no more a LISP philosophy than a C++ one. The difference is that way more people are working with C++ (or Java) thus opensourcing their libraries for others to be used.
Deleted Comment
I am pretty sure you are expected to know how to program if you use any programming language. And writing everything from scratch, instead of taking advantage of the work done by thousands of other skilled developers, is just silly. With that attitude, no wonders the world runs on C/C++ and not Lisp.
If you are every in need of a subject to write about--I might even pay for a walkthrough of the code that built Hackernews.
I don't think I'm alone either.
Deleted Comment
Do libraries exist in Lisp on the same scale as they exist in Java? Especially, do they exist with the same level of quality, professionalism, and maintenance as they exist in Java? I'm guessing that the answer is no.
Lisp isn't very popular because, at least in my opinion, it requires a relatively high level of maturity and experience to be used effectively on nontrivial projects.
Every single project in Lisp (Common Lisp, Clojure) that I have observed ended in a disaster mostly because developers could not figure out what the strengths of Lisp are to realize the benefits and then killed themselves with too much freedom.
Absent large enough number of success stories companies will default on Java because it is good enough language to be able to effectively realize typical enterprise application and dumb enough to be used by a beginner dev, even at the cost of boilerplate.
At least that's the official story. It is really difficult to explain to business that you could do whatever they want with 1/5th or 1/50th of the amount of code if you just used a good language for it.
Other languages scale smoothly from beginner to expert. So I guess that your assertion shows that Lisp is a less useful language?
Experienced programmers using non-Lisp languages are successful every day. Your OS/mobile phone/browser/… is the result of their work. So I guess other languages are better for experienced developers as well?
In other words, the real world evidence seems to suggest that Lisp is not good for beginners and not good for experienced developers.
Dead Comment
Things are not so simple
[1] talking about speed