Readit News logoReadit News
Posted by u/graderjs 3 years ago
Ask HN: Why are JavaScript dependencies so messy?
I love JS, but every once in a while a new bundler comes along that "solves everything". And it works, for a while. then it breaks. Why? Why are there so many edge cases? I don't understand it. We only have a few module types (AMD, CommonJS, ES modules), with a few types of import and export syntax. How hard can it be to get it always right?

Like parcel. It worked. For a while. And now if you check the GitHub there's 690 open issues, and I had issues today getting it to work when running after an 'npm i' done in v17 or v18, yet it's fine to run in v{16,17,18} if 'npm i' is done in v16.

And snowpack: v0 (or 1) worked great, but the next version broke so many things (compared to the prior version) that I need to keep the dep version locked to the earliest ones for packages where I use that. Tho I guess that's more of an API problem.

What I'm really talking about is: why can't we just have a bundler that works always and everywhere (and I don't want to 'wait for' deno)?

Why would parcel start to get bugs...how hard can it be??? :...(

samwillis · 3 years ago
I think there are a few things here that are kind of unique to the js/web platform.

- JS doesn’t have a proper standard library and so you tend to have many more dependancies in a project than with other languages.

- Because of that you then have a very large tree of dependancies, often with incompatible version requirements of the same package.

- Due to the nature of needing to “bundle” your code for distribution on a “slow” network, a lot of optimisations (tree shaking, code splitting, async module loading) are required to minimise your bundle size.

- You say “We only have a few module types”, that’s two more that every other language. And they are incomparable in subtle ways.

Those and some other issues are completely unique to the web platform, they are some difficult problems to solve.

However, I think the current generation of tooling has finally got there. Vite, with its esbuild and Rollup backend is bloody brilliant. If you use a framework with official Vite support it just works.

throwaway0asd · 3 years ago
There have been many attempts at standard libraries that are always extremely opinionated and often introduce performance penalties and an additional layer of defects. That has nothing to do with the tremendous number of dependencies that would exist anyways if you aren’t extremely aggressive about dependency management.
fuzzythinker · 3 years ago
Second "Vite (esbuild) just works". I would also recommend start using bun.sh on smaller projects/scripts so you have first hand knowledge of how ready to production it is. Jarred works in an insane mode, so I wouldn't be surprised if it turns get mass adaption before deno does.
Bellend · 3 years ago
Why doesn't Javascript have a standard library? As in, why has one never been made and shipped with the browser?
krapp · 3 years ago
Javascript has a standard library. What Javascript doesn't have is a robust kitchen-sink library akin to other more complex general purpose programming languages, because Javascript was designed to be used in the context of a browser to script webpages, and its actual standard library reflects that.

Unsurprisingly, because we've apparently decided Javascript needs to replace all existing languages in all current and future forms of application and system development, its actual standard library is no longer sufficient for many peoples' requirements. But they would rather claim the language "has no standard library" in order to justify the insane complexity of the JS ecosystem than admit that perhaps it just isn't the right tool for every job.

unsafecast · 3 years ago
Oh, they ship stuff akin to one, but that's the web APIs, not a “standard” library. As in it's not really standard.
gw99 · 3 years ago
I'm sure I'll get downvoted for this one but quite frankly it's where the PHP developers all slithered off to. It was inevitable.

I avoid to the point of refuse to work with any JavaScript platform at all. They are a universal shit show.

Edit: honestly I stand by this comment. I literally have spent hours this year dealing with fucked up messes in nodejs, package security issues after repos were hacked and code made it into NPM that displayed banners on commercial sites. My comment is toxic yes but quite frankly both communities are as well (PHP and NodeJS) and utterly earned that ire and discontent. The fact people lean on the stack and produce stuff that handles critical aspects of people's lives is utterly frightening. So yeah toxic, I don't care. Feel free to be annoyed. It's still a turd, just been rolled in glitter.

jef_leppard · 3 years ago
> quite frankly it's where the PHP developers all slithered off to

PHP isn't my favorite language but using phrasing like "slither" to describe an entire community of developers is frankly shitty. Every language has its problems including whatever you use.

gw99 · 3 years ago
I spent most of 2005-2013 doing contract work getting people out of near business ending buckets of shit thanks to the PHP community's inability to produce a product that was fit for purpose and the awful staff contracting building on top of the wobbly pillars of excrement. Slither is exactly the word because they were the first people to slither away and point fingers elsewhere when something was wrong. Now ten years after that I am in regular contact with people actively getting rid of it and it's the same problems. And here I am now on Sunday evening eviscerating another failed nodejs stack left by the same slithering fiends.

Oh no it's exactly right. Slithering is the word.

I don't get anywhere near this level of sheer incompetence on a daily basis in the Python, .Net or Go ecosystems. Nor do I have problems sleeping wondering what shit show am I going to face on a Monday morning with them.

sli · 3 years ago
I always say the reason PHP is still so hated is because rather than arguing about it on the internet to defend the language, PHP programmers are going home to their spouse, kids, and hobbies at the end of the work day.
aliqot · 3 years ago
Unless Python, then slither is fine.
Nextgrid · 3 years ago
Not sure about the developers, but PHP and Javascript have a problem in common:

PHP has some sort of standard library but it's not intuitive (the function naming is a mess), sometimes outdated, wrong or broken (but the broken behavior has now ossified and will never be removed for backwards-compatibility reasons).

JS has basically no standard library at all.

Both of these issues mean everyone has to use third-party packages to make up for that, of which there are tons, each with their own issues, breaking changes, upgrades, etc.

dawnerd · 3 years ago
JavaScript has a ton built in but developers get in a habit of using libraries that introduce their own api and polyfills on top. I blame awful browser support of years past. Today most new features are available pretty quickly across all the major versions.
onli · 3 years ago
PHP changes behaviour of core functions all the time now (and subsequently breaks old programs with every release). In every release stuff is deprecated and deprecated stuff removed. Look at https://www.php.net/manual/de/function.implode.php for a telling example and check the 8.1 changelog at https://www.php.net/releases/8.1/en.php for a list of bc breaking changes.
lucideer · 3 years ago
The irony of this comment is that Composer & packagist is one of the sanest and most well-thought-through package management systems. PHP actually fares better than the vast majority of language ecosystems on the specific topic being discussed in this comment thread.
girvo · 3 years ago
Although Composer was incredibly slow. No idea if that’s still true, but I believe it was partially a consequence of its approach to dependency constraints, which is what made it actually quite robust.
biryani_chicken · 3 years ago
PHP's dependency management is actually quite nice. The community developed a standard [0], a client [1] and repository [2] that pretty much deal with everything I've had to do with PHP dependencies without issues.

[0] https://www.php-fig.org/psr/psr-4/

[1] https://getcomposer.org/

[2] https://packagist.org/

gw99 · 3 years ago
So you can use nice tools to bake a poop cake :)

That's not particularly pointed at PHP that. There is a big problem in this industry about where we get our software from and how it's built and managed but that's well outside the topic of this thread. Fundamentally everything is fundamentally broken with dependency management, trust and competence.

dawiddr · 3 years ago
Those might not be the same people, but for many people JS is the door to the world of software development - the same way PHP was (also for me).
SquareWheel · 3 years ago
> I'm sure I'll get downvoted for this one...

Rightfully so. Your comment is utterly toxic.

aliqot · 3 years ago
JS is retracing some of the same steps of Python. They are both a fun language with not a lot of constraints and a lot of capability.

What this does is create a large community of enthusiastic beginners led by few (prone to follow trends by authoritative members) so you get some experts but a lot of beginners and beginners love libraries, especially ones written by the expert contingent. The JS stdlib is not as strong as it could be, nor is python, so libraries in some cases make a lot of sense but for beginners it is easy to rely on libraries for everything.

Eventually some of these beginners turn into skilled programmers and either stick with just JS or Python, and then the other subset moves on and tries other languages that address some of the perceived 'shortcomings' of JS/Python. What you find is that a lot of the ecosystem becomes that beginner crew and a repeated brain drain of the other 70% who move on.

LtWorf · 3 years ago
> The JS stdlib is not as strong as it could be, nor is python

Comparing the js stdlib with python one tells me you never used python.

Also the theory that experts move on while noobs stay is just a theory. I'm going to assume you personally moved to rust and need to call yourself an expert.

LtWorf · 3 years ago
Seems people aren't liking this.

However the average python project has 25 dependencies and the average js has 174 (source https://snyk.io/reports/open-source-security/) so perhaps js and python do not have equivalent stdlib?

phailhaus · 3 years ago
Dependencies is different from bundling. Javascript's dependency management is an absolute dream, and you can have a newbie up and running with a consistent environment very quickly. Compare that to Python, which still doesn't have a declarative package manifest format (leading to extremely slow/inconsistent package resolution), or a mature "lockfile" for deterministic installs. npm has both of these out of the box. You can even assert what versions of npm or node you need in your package.json to make sure you're using known-working versions.

EDIT: Turns out Python just finalized a pyproject.toml format in 2021. Of course, this doesn't really help much until every package out there migrates. npm has used the package.json format pretty much since day one. And there is still no standard lockfile, other than dumping the output of `pip freeze`.

plonk · 3 years ago
> Compare that to Python, which still doesn't have a declarative package manifest format

It does, with pyproject.toml [1]. It even had two [2], but setup.cfg is deprecated because it was specific to setuptools.

[1] https://setuptools.pypa.io/en/latest/userguide/pyproject_con...

[2] https://setuptools.pypa.io/en/latest/userguide/declarative_c...

isoprophlex · 3 years ago
About the situation in python... Have you tried using "pip freeze > requirements.txt" to get a deterministic set of requirements? Or am I thinking too simplistic?
phailhaus · 3 years ago
Yeah, that's unfortunately not good enough. Ideally, you want to store the graph of dependencies so that you can understand why each package is being installed, with useful metadata like the hash of package to ensure you're getting the same code.
wink · 3 years ago
Just because Python's story here isn't good doesn't mean JS's is automatically good.

Your environment may be consistent at this point in time but it's still a horrible mess.

phailhaus · 3 years ago
How so? OP's gripes were mostly around bundling, but the dependency management in JS is pretty solid.
chimen · 3 years ago
Because it's a language and ecosystem built by mostly designers and unicorns not backend programmers who rely more on logic and functionality rather than colors and rainbows. My 2c.
_pdp_ · 3 years ago
I tried to compile a rust project the other day and I realised I had to download hundreds of dependencies. I guess strong typing helps but dependency hell is just common for any moderately popular platform.
aliqot · 3 years ago
To be fair a lot of the JS developers that wanted something better moved to Rust, and brought a lot of the same ethos with them. Anybody who's looked at the import list to their favorite Rust app has probably said to themselves "woah its just like a node app". Not saying it's a bad thing, I'm more of a stdlib guy but I see how it grew to be the way it is.
CoolestBeans · 3 years ago
With both Rust and node, it isn’t easy to do the things that should be easy. So developers will take the path of least resistance to make it easy. And both Rust and node make their package manager their project manager so everyone is an “npm i” or a “cargo add” away from making their lives easier.
gdprrrr · 3 years ago
I'd say that's better then the C way, where every projects Implements their own version of a linked list, json parser and so on, no?
Gigachad · 3 years ago
C hides all the dependancies. Rather than seeing the 100 sub dependancies, they just tell you to download the binary of libthing where the sub dependancies are prebaked in so it looks like you have 4 dependancies.
simmerup · 3 years ago
I hated NPM... then I started trying to use some Python projects.

Now I like NPM.

lucideer · 3 years ago
Yes!

I feel like so much criticism of NPM comes from either people who've only ever used NPM, or people who haven't used NPM and just saw a big node_modules once.

A common one in work is Java devs criticising the huge node_modules folder but not realising that .m2 exists as their IDE handles it for them "by magic".

simmerup · 3 years ago
To be fair I prefer Maven, but probably more for the culture. In Node you generally ask for the latest version and hope it’s still available/the project still builds when you come back to it. pom files are more clunky but they do tie you down more
waste_monk · 3 years ago
How is python package management a problem? spin up a new venv, pip install whatever, away you go. I don't do a lot of python work so maybe I'm not hitting the boundaries, but it's never been anything less than pleasant to work with in my experience.

JavaScript can go directly to hell.

pyuser583 · 3 years ago
Dependencies are hard because dependencies are hard.

There’s no easy dependency management system.

Bash - the most interoperable language in existence - contains its own version of malloc because some systems used to need that (back 30 years ago).

JS has an extremely uncertain runtime. Is it in a browser? Engine? PDF?