Readit News logoReadit News
oblio · a year ago
matklad · a year ago
And

https://rust-analyzer.github.io/blog/2020/07/20/three-archit...

Is a must-read if you are to build your own LSP server!

sestep · a year ago
That 2022 post is amazing, thanks for the link!
ogoffart · a year ago
I've implemented a LSP server (for Slint - https://slint.dev) and I agree with this article.

The paradox is that it was meant to avoid to write language support for each editor. Yet, if you want to support vscode you must create a specific extension for it and can't just have a language client.

The article mention the configuration problem, but I'd add the problem that Microsoft refuses to specify a way for the server to tell the client what are the config options so that the client can show some kind UI showing the possible configuration options with a description of what they do. https://github.com/microsoft/language-server-protocol/issues...

dualogy · a year ago
> The paradox is that it was meant to avoid to write language support for each editor. Yet, if you want to support vscode you must create a specific extension for it and can't just have a language client.

On the one hand, this has always bugged me slightly. But on the other hand, every time when it came down to it in practice, I realized two things: first, a VSC extension that does nothing but wire up the LSP language clienting library is a tiny project with no major maintenance burden; and secondly, more importantly you usually want additional language-extension client-side features that are editor-specific by nature and either couldn't or shouldn't be abstracted-inside / covered-by the language-server / lang intel / IntelliSense realm. Syntax-coloring and other grammar-describing meta-data come to mind, or custom Notebook, or rich & productive build or debug or pkg-mgmt etc. helpers & tooling UX, etc. Stuff that arguably belongs in one's "language extension" but is mostly editor-specific and thus at the same time doesn't truly belong in one's "language intelligence serving".

ReleaseCandidat · a year ago
> Syntax-coloring and [...] come to mind

Semantic highlighting is part of LSP and supported by all "bigger" LSPs.

blahgeek · a year ago
> Yet, if you want to support vscode you must create a specific extension for it and can't just have a language client.

It’s funny that you generally don’t need to do that for editors like vim or emacs - only need to add a single line in config to specify the command line argument

bryjnar · a year ago
It depends - the client modules for Emacs' `lsp-mode` generally need a fair bit of configuration. Not just how to launch the server, but also often defining a native-compatible way of setting the options. e.g. for Emacs they often get wired up as `defcustom`s.
ratmice · a year ago
It honestly depends on how much your LSP server infests the editor. In the language server I wrote there are really 2 cases where we utilize the creation of an editor-specific extension

* dynamic registration of file extensions * display of svg generated by the language server.

We also had to write ad-hoc custom vim-script and would need to do the same for emacs for the first of those, and just dump a URL for vim to punt to a browser for the latter. But it isn't unrealistic to require a custom editor scripts for other editors besides vscode, in the sense that I've done so...

paxys · a year ago
That's because the LSP makes no assumptions about stuff like (1) what the language server is, (2) how it is launched and managed, (3) how the editor communicates with it, etc. It only defines the format of the communication, nothing else.

In the simplest form the entire language extension in VS Code can be ~15 lines of code that forks the server process and talks with it over stdio. But in complex cases you can:

- Write the entire language server in the extension itself.

- Have your code hosted on a different machine or Docker container.

- Run the editor in a browser and get language support though a service worker or a TCP connection to a remote server.

LSP supports all of this seamlessly because it is agnostic about the transport layer.

ogoffart · a year ago
Other editors have kind of standardized it to associate a language to a command to run and communicate trough the stdout/stdin. Why do vscode need to be more complicated?

(even this association could be done with a local DB similar to mime types if we wanted to simplify)

aidenn0 · a year ago
RE: "Not a truly open project."

If LSP isn't truly open, then neither are most GNU projects. It was very common for the first 15+ years of GNU's existence for the public development process of a project to be "the single maintainer publishes a release archive whenever they feel like it"

It's a standard freely published and available for all to implement. If that's not "truly open" then we have moved the goalposts way too far.

jchw · a year ago
I think "truly open" is not specific enough. Not being developed "in the open" is one thing, not having "open governance" is another thing.

That said, I guess the problem here is that for standards it helps if well, you collaborate with the people for which the standard is meant to be used by, which is presumably a little hard if there's a huge asymmetric relationship when it comes to Microsoft's concerns vs the rest of the world's concerns.

This is one of those cases where having a standards committee or consortium is the way to go. Committees have their problems, but I think it's only reasonable. If you think about it, doesn't it seem inevitable that eventually, big organizations that make editors would want a consortium of some sort to collaborate on protocols like this? LSP is really just the beginning, since there are plenty of things that editors would probably like to integrate deeper with, such as build systems.

aidenn0 · a year ago
> This is one of those cases where having a standards committee or consortium is the way to go. Committees have their problems, but I think it's only reasonable.

I think a committee is a reasonable backup plan, and can even be done without Dirk's approval (see e.g. WHATWG done without the W3C's approval). If the LSP continues to be "good enough" then it seems unlikely.

> If you think about it, doesn't it seem inevitable that eventually, big organizations that make editors would want a consortium of some sort to collaborate on protocols like this?

Maybe? Editors tend to be an afterthought for most companies; JetBrains is the only company I can think of that is big on the LSP and for whom the editor is a primary experience.

ReleaseCandidat · a year ago
> doesn't it seem inevitable that eventually, big organizations that make editors would want a consortium of some sort to collaborate on protocols like this?

Jetbrains' main feature are their own engines. And MS still has Visual Studio - without Code ;) - which doesn't use LSP. And Apple is Apple. So, who would that be? On the contrary I'd say that everybody who want's to get "big", must not use LSP to have something that sets the editor apart.

kelnos · a year ago
Two things:

1. (As another commenter mentioned) Most GNU projects are not standards that are expected to be adopted by a significant number of implementers, and used by a huge number of users. Most GNU projects are totally fine having a few maintainers.

2. I am a lot more comfortable with a GNU project being run by a single maintainer than a public specification being owned by a corporation, where changes to that specification are largely driven by that company's product choices and profit motive.

And regardless, it seems a little weird to compare the GNU of the 80s and 90s to any public project today. In GNU's first 15 years the internet was nascent (at best!), and the number of people who implemented, used, and cared about these sorts of things were orders of magnitude smaller than they are today. Needs have changed.

aithrowaway1987 · a year ago
But that is not the standard for current GNU projects in large part because of all the easily avoidable friction. "If it was good enough for Richard Stallman in 1987, it's good enough for Microsoft in 2024" is just a dumb argument.

Not to mention you're conflating apples with oranges, since a software standard is very different from an application. POSIX wasn't just one Bell Labs employee working by himself.

From the article:

> The LSP should be an open standard, like HTTP, with an open committee that represents the large community which is invested in LSP, and can offer their insight in how to evolve it.

There is no goalpost moving here.

cryptonector · a year ago
Building and maintaining a community is hard work. Even just talking to all comers is hard work. You need a team for that, but if you're a team of one then the community is likely going to suffer. You could find external contributors to promote to committers, but that's work too, and maybe LSP's maintainer doesn't want that (or maybe LSP's maintainer's employer (MSFT) doesn't want that). Apart from what MSFT wants, the rest is just as likely to happen for small enough projects whether they be GNU projects or not.

Deleted Comment

leni536 · a year ago
Most GNU projects are typically not standards/specifications but programs and libraries. That's a significant difference IMO.

Deleted Comment

kayodelycaon · a year ago
Yeah… this is can be confusing. “Open source”, “open standard”, and “open project” are different concepts.

The first two are well-defined and LSP meets the criteria for both.

“Open project” doesn’t have a definition or criteria. In this case, it probably means the community using a standard, controls the standard. The http protocol is an example of this.

Ultimately, they think the people in charge of the project are not listening to the people using it. This is a serious concern.

However, making a project “open” does not fix this problem. See systemd.

I hate dropping Wikipedia links but here is open standard: https://en.m.wikipedia.org/wiki/Open_standard

Deleted Comment

forrestthewoods · a year ago
LSP is pretty ok. Better than the before times I suppose. Although C++ has long had good IDE support so it hasn't affected me too much.

I have a maybe wrong and bad opinion that LSP is actually at the wrong level. Right now every language needs to implement a from scratch implementation of their LSP server. These implementations are HUGE and take YEARS to develop. rust-analyzer is over 365,000 lines of code. And every language has their own massive, independent implementation.

When it comes to debugging all native language support common debug symbol formats. PDB for Windows and DWARF for Nixy things. Any compiled language that uses LLVM gets debug symbols and rich debugging "for free".

I think there should be a common Intellisense Database file format for providing LSP or LSP-like capabilities. Ok sure there will still be per-language work to be done to implement the IDB format. But you'd get like 95% of the implementation for free for any LLVM language. And generating a common IDB format should be a lot simpler than implementing a kajillion LSP protocols.

My dream world has a support file that contains: full debug symbols, full source code, and full intellisense data. It should be trivial to debug old binaries with full debugging, source, and intellisense. This world could exist and is within reach!

aidenn0 · a year ago
1. I use zero languages that use either PDB or DWARF that are not named "C".

2. You are either overestimating the level of detail available in PDB/DWARF or underestimating the massive amount of language-specific work needed for even basic features (e.g. methods, which lack any cross-language ABI) given just what PDB/DWARF give you.

3. What LSP provides and what PDB/DWARF offer are only very loosely related. Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF.

forrestthewoods · a year ago
> You are either overestimating the level of detail available in PDB/DWARF

Uhhh. I didn’t say PDB/DWARF already have the necessary information. In fact I even proposed a new file format! I suggest you re-read what I said.

> Consider the case of writing function1, then (without compiling) writing function2 that calls function1. It is typical for an LSP to offer completion and argument information when writing out the call for function1. That's not something you get "for free" with PDB/DWARF.

What do you think LSP servers do in the background? They’re effectively compilers that are CONSTANTLY compiling the code.

Amusingly rust-analyzer takes longer to bootstrap than a full and complete clean and build. Maybe it’s not as parallel, I’m not sure.

PoignardAzur · a year ago
> rust-analyzer is over 365,000 lines of code.

That has nothing to do with LSP.

Rust Analyzer is similar in scope to a second implementation of the Rust compiler.

czei002 · a year ago
I would blame Rust though. For example, Rust has macros which are way too powerful and make it very hard to write a LSP (https://rust-analyzer.github.io/blog/2021/11/21/ides-and-mac...)

Very interesting is how Roslyn/Typescript does it: https://www.youtube.com/watch?v=qnyOHY7AiZk

forrestthewoods · a year ago
> similar in scope to a second implementation of the Rust compiler.

I know. That’s really bad!

ReleaseCandidat · a year ago
Rust-analýzer is an example of what not to do, which is reimplementing a compiler frontend. Ideálly it should be the samé as the "real" compiler is using. Of course this has it's own problems, which the Haskell LSP this post is about, shows. As compilers not written for being used "interactively".

> Any compiled language that uses LLVM gets debug symbols and rich debugging "for free".

That doesn't hold for C++ and much less for any language even "less C" than C++. Like languages using a GC, e.g. Roc https://www.roc-lang.org/

forrestthewoods · a year ago
> That doesn't hold for C++

What do you mean? Why not? Clang PDBs for C++ work great. A GC isn’t particular disruptive to debug symbols afaik.

Zababa · a year ago
"any LLVM language" is a lot but also not that much. You're missing Python, JS, Go, Ruby, etc.
thramp · a year ago
I've responded on reddit before (https://www.reddit.com/r/rust/comments/1eqqwa7/comment/lhwwn...), but I'll restate and cover some other things here.

> I have a maybe wrong and bad opinion that LSP is actually at the wrong level. Right now every language needs to implement a from scratch implementation of their LSP server. These implementations are HUGE and take YEARS to develop. rust-analyzer is over 365,000 lines of code. And every language has their own massive, independent implementation.

rust-analyzer a big codebase, but it's also less problematic than the raw numbers would make you think. rust-analyzer has a bunch of advanced functionality (term search https://github.com/rust-lang/rust-analyzer/pull/16092 and refactors), assists (nearly 20% of rust-analyzer!) and tests.

> I think there should be a common Intellisense Database file format for providing LSP or LSP-like capabilities. Ok sure there will still be per-language work to be done to implement the IDB format.

I think you might be describing formats like (https://code.visualstudio.com/blogs/2019/02/19/lsif) and SCIP (https://github.com/sourcegraph/scip). I personally like SCIP a bit more LSIF because SCIP's design makes it substantially easier to incrementally update a large index. We use SCIP with Glean (https://glean.software/) at work; it's pretty nice.

> But you'd get like 95% of the implementation for free for any LLVM language. And generating a common IDB format should be a lot simpler than implementing a kajillion LSP protocols.

I wouldn't say 95%. SCIP/LSIF can do the job for navigation, but that's only a subset of what you want from an IDE. For example: - Intellisense/autocomplete is extremely latency sensitive where milliseconds count. If you have features like Rust/Haskell's traits/typeclasses that allow writing blanket implementations like `impl<T> SomeTrait for T`, it's often faster to try to solve that trait bound on-the-fly than storing/persisting that data. - It'd be nice to handle features like refactors/assists/lightbulbs. That's going to result in a bunch of de novo code needs to exist outside of a standard compiler, not counting all the supporting infrastructure.

> My dream world has a support file that contains: full debug symbols, full source code, and full intellisense data.

Rust tried something similar in 2017 with the Rust Language Server (RLS, https://github.com/rust-lang/rls). It worked, but most people found it too slow because it was invoking a batch compiler on every keystroke.

kayodelycaon · a year ago
> The LSP specification is big. Really big. Last time I checked it had 90 (!) methods and 407 (!!) types. Printing it to a PDF gives you 285 pages (!!!).

Given what it does, that’s pretty dang small. You don’t get all those features across every programming language without some complexity.

Compared to the 3,000+ pages specifications I’ve dealt with, this is easy.

I have a very tiny violin and a cricket to play it for anyone complaining about a few hundred pages. :)

saghm · a year ago
What sticks out to me is that there's less than one method per every four types. Given that, I assume that most of these types are just plain structs or type aliases; in other words, just names for things. I'm not sure what the author is trying to convey with that; would it be better if there were half as many types but they all had twice as many fields and methods and had to get used for multiple purposes?

If the scope of the project is too large, that's certainly a valid complaint, but independent of somehow measuring the expected "size" of an API to cover a given scope, I don't really see how these raw numbers are particularly concerning. I'd be more concerned if the ratio of methods to types were reversed, but still a bit skeptical of any significance without more context.

diggan · a year ago
> would it be better if there were half as many types but they all had twice as many fields and methods and had to get used for multiple purposes?

Reminds me of this quote:

> 9. It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.

Written by Alan Perils, shared in "Epigrams in Programming" (1982): https://cpsc.yale.edu/epigrams-programming

Maybe I'm lisp-damaged, but I agree with this quote and parent, more functions (not methods) and less data structures tend to be easier to manage when things grow in scope.

kayodelycaon · a year ago
For comparison, YAML is 77 pages.
Uehreka · a year ago
I was gonna say, I’ve had to pore over 800+ page PDFs from the 3GPP describing like, the possible fields for one particular protocol in the LTE spec. 285 pages for the whole shebang feels Hemingwayesque.
sph · a year ago
Your argument basically boils down to "I am cooler than you because I have worked on larger specs". The fact that there are hilariously larger documents doesn't offer any perspective on whether you need 300 pages to describe how to color source code in a text editor.

But good for you! Here's a very tiny award.

kayodelycaon · a year ago
In all seriousness, I do have a point. I’ve read enough documentation to know the size of LSP’s documentation is what I would expect for something of that complexity.

The author’s surprise speaks more to their unfamiliarity with projects at this scale than anything special about LSP.

And I had to be sarcastic about it. That’s why I had :) at the end of my comment.

As far as being cooler, I’m not. If anything the author is way cooler. I’m just a random snarky dog on the internet. :)

iudqnolq · a year ago
I pulled up a random section and saw a method the editor sends right before a save so that the lsp can do format on save. I think format on save is pretty handy. Would you cut it?

(There's two methods, one with a timeout. In an ideal world maybe that would be one but I totally understand how initially they maybe assumed format on save would always be reasonably quick and then had to backtrack)

kelnos · a year ago
> So I don’t think it’s really a good idea to do a big re-engineering of the protocol just to make it easier for implementers

I'm not so sure. Maybe it's not the case today, but if the protocol eventually becomes so large, so crufty, so under-specified that implementers struggle to provide their implementations, then users suffer.

csb6 · a year ago
The article touched on it, but it would be nice if there were a standard way to have extensions for LSP, e.g. a protocol that could be shared by language servers for formal specification languages that need proof obligation support. [0]

[0] http://dx.doi.org/10.4204/EPTCS.338.3

sesm · a year ago
The problem that Microsoft is solving is promotion of VsCode and Visual Studio. Making LSP small and easy to implement would go against that. They want LSP to work well only in their IDEs. Yes, this makes implementing LSP for languages harder, but most of those developers are paid by MS directly or indirectly, in fact making LSP hard also works as a filter against unpaid contributors.
zarzavat · a year ago
When it comes to IDEs, small and easy = uncompetitive.

Most users want to use the IDE that saves them the most time and has the features they want, they are not concerned with implementation complexity because that’s someone else’s job.

If you reduce all the complexity of a competitive IDE to a protocol then you get something as complicated as LSP. That’s just how it is.

eropple · a year ago
Having read the LSP specification, and when I take into account just how many things LSP does, I think a 285-page spec is really tiny. I have specs on my work computer with TMF and CAMARA specs that are nearly 100 pages just to talk about the operation of a half-dozen methods--and there's still ambiguity in them at times.

Coupling that genuine brevity with the ability to avoid serious backwards-compatibility problems makes this charge feel pretty outlandish, TBH.