Readit News logoReadit News
klodolph · 9 years ago
It seems like a lot of JavaScript developers are repeating things like "more people know JavaScript so you don't have to learn a new language, which saves you time". I don't get it. In my experience, if you find a good developer, they can pick up C#, Swift, or Go pretty quickly, and if you can't find a good developer, the fact that they already know JavaScript is not much of an advantage. Even if the developer you hire already knows your language, they're going to be spending time learning your code base and how your organization works (shared repo? PRs? Feature branches? Code review? Coding standards?)

That, and nose.js developers seem to repeat the claim that node.js makes delivery faster… but is it really any faster than ASP.NET, Rails, Django, or Go stdlib? Those frameworks are so fast for prototyping and delivering bread-and-butter apps as it is (and some of them let you do multithreading to boot).

I'm also really not interested in how things work for "typical CRUD apps" because those are so trivial to write in any decent environment.

I'm worried that node.js articles are the same kind of echo chamber that Rails articles were 10 ago.

jeswin · 9 years ago
The language doesn't matter, but the ecosystem does. It is when it comes to "non-CRUD apps" that an ecosystem will really start to matter. One is probably not better than the other but getting started is hard.

Take the case of Node and JavaScript itself. The ecosystem is huge. The tooling is complex. A newbie starting out will not even be able to get many examples to run without a mesh of transpilers and build tools. There are roadblocks everywhere; what's the type annotation there? What's with the strange JSX syntax? What's a transpiler? Add to that, EcmaScript itself is a moving target, more so than any other language.

Similar issues exist in other languages. C# is a simple language to master, .Net is a huge ecosystem. There's not going to be a lot of resistance moving from C# to F#. But for the newbie, the challenges are daunting. Should you use Microsoft's preferred ORM which ships with .Net or write your own SQL? Which UI tooling should I use? There is plenty of advice on the web, yet it needs an experienced developer to make those decisions.

Consider this. if you chose SilverLight a few years back, your product is soup today.

Picking an ecosystem is not easy. Has never been.

TAForObvReasons · 9 years ago
> Consider this. if you chose SilverLight a few years back, your product is soup today.

The amazing part about JS is that most of the code written for browsers a decade ago still works today. It is extremely unlikely that we will see a company outright ban JS in their web browsers in the same way that iDevices don't support flash.

weberc2 · 9 years ago
I will say that Go makes the default path pretty painless. If you can conform to the directory structure, you can build a binary for any platform from any platform with a single command--no metadata files or build scripts. From there, deployment is a matter of moving a single binary. No interpreter, no VM, no runtime dependencies. It's opinionated nature can be annoying, but it means a lot of choices are made for you. I'm not familiar with another language that offers such a fast path.
mgkimsal · 9 years ago
> I'm worried that node.js articles are the same kind of echo chamber that Rails articles were 10 ago.

I tend to agree.

> That, and nose.js developers seem to repeat the claim that node.js makes delivery faster…

The qualification was "if your team already knows JavaScript". OK... If the team already knows Java... building something like this in Java would be faster to deliver, and wouldn't hit some of those issues the author brought up (4 vs 1, single vs threaded, etc).

And the whole "oh, you know language X" - means almost nothing when the project is more than trivial hello world. Every project I've been brought in on, the qualifier was "must know tech X". And almost always (there were exceptions) I knew more about tech X and software dev in general than the original developers, and the crunch was not "how do I do XYZ in this tech?" it was "how do I get the other developers to actually use version control?" or "use version control sanely?" or "document anything?" or "write tests" or "have test data" or "have a repeatable build process"?

Knowing ASP or PHP or Ruby or whatever, but going in to a project without repeatable build process, tests, documentation, requirements or version control, is a recipe for disaster. Stressing the "deliver quickly" aspect of any language, if you're actually trying to deliver for a business or with a team of people, is extremely destructive short term thinking. And yes, sometimes, in rare cases, it may be a necessary evil, but I think it's become the norm with words like "agile" being thrown around as synonyms for "don't have to write anything down".

ekidd · 9 years ago
> Every project I've been brought in on, the qualifier was "must know tech X". And almost always (there were exceptions) I knew more about tech X and software dev in general than the original developers, and the crunch was not "how do I do XYZ in this tech?" it was "how do I get the other developers to actually use version control?" or "use version control sanely?" or "document anything?" or "write tests" or "have test data" or "have a repeatable build process"?

I've seen this so many times on consulting gigs and with the occasional new employer. I'm hired because I'm familiar with the technologies they're trying to use, but the first major value I bring to the organization is simply getting some kind of sane process: working version control, automatic builds, test suites, one-button deploys, and so on.

halisaurus · 9 years ago
> Stressing the "deliver quickly" aspect of any language, if you're actually trying to deliver for a business or with a team of people, is extremely destructive short term thinking. And yes, sometimes, in rare cases, it may be a necessary evil, but I think it's become the norm with words like "agile" being thrown around as synonyms for "don't have to write anything down".

Unfortunately, this is pervasive in the "real world", especially with regard to client-driven (agency) work. Deadlines beat developers almost without fail. I agree that it is destructive short term thinking. Entire weeks of programming can be destroyed in a 15-minute phone call that (re)highlights a limitation clients either didn't account for in their specification (if you can call it that) or just chose to ignore.

This isn't to say that using a language or framework a developer is familiar with is baseless, but I've heard this advice before and see it reinforced often:

1 - Hire people not skills.

2 - Success is 90% preparation.

astinit · 9 years ago
As someone still studying and not in the workforce yet, are there really teams out there that don't use version control?
lordofmoria · 9 years ago
"Must know tech X" is exactly the mentality that we try to avoid in our company's hiring process. Most people would claim this makes your job descriptions too flexible, but the demand is actually incredibly specific: developers who can jump on a project in a tech they haven't "used" before and succeed 90% of the time. It's probably the case that only 10% of devs can do that comfortably though. Fits the criteria in 'Who' perfectly!
smokeyj · 9 years ago
> I'm also really not interested in how things work for "typical CRUD apps"

You mean you're not interested in what a huge part of the industry is doing? Well then that explains why you don't see the benefit.

There's tons of work creating user interfaces, transform some business logic and shove it back in a DB. No one said it was going to be a life-fulfilling experience. It's just work.

Now many of these shops want a native ui, mobile ui, responsive design and a back-end developer all in one. Many times they hire for specific frameworks so that you're ready to be productive day 1.

If I were staffing up a team to do a CRUD app, or even more complex, I'd staff up with JS developers.

noir_lord · 9 years ago
I like doing CRUD apps, beyond the trivial you run into a terrific amount of business logic and thats an opportunity to really have an impact, one of my proudest moments as a programmer was taking a process that was largely manual and took 3 staff 2 days and making it something the boss could do in half an hour on a Friday afternoon by redefining the underlying assumptions about how their business worked - it's amazing how many businesses do <involved complex manual process> because that's how we've always done <involved complex manual process>.

It's 2017, We've had desktop computers for 35 years and most of the people I know working in various industries still manage a huge chunk of their work in email and excel because it's hard for their management to understand the benefits of automation or why they'd make an investment in the longer term with good software, I know of one company thats in the FTSE250 that manages it's entire pallet and tray inventory by using excel spreadsheets on a shared drive and sending people from the office to sites to look whats there (they order 100K+ Trays at a time).

klodolph · 9 years ago
> You mean you're not interested in what a huge part of the industry is doing? Well then that explains why you don't see the benefit.

Yes, you chopped off the important half of the sentence, and responded as if I hadn't written it. I said that I wasn't interesting because CRUD apps are already so easy to write in many different frameworks.

Let's assume that what you're saying is true—that most shops want all of those skills rolled into one person. The people I've met that are good at both front-end and back-end development are comfortable in multiple languages to begin with. The developers who only know JavaScript and are not comfortable in other languages are not good full stack developers.

But I don't believe that this is true to begin with, most places I've seen or the places where my friends work have people who focus on front-end and people who focus on back-end. There's just too much of a crazy front-end landscape to expect magic from someone who doesn't live in that world, and back-end is its own world.

rb808 · 9 years ago
I've worked on projects in which both the front and end back end were written in the same language & those that are different. The former were more productive imho with developers working on both and with shared libraries & shared test platforms. The latter often end up getting silo'd with stuff passed over the fence. Yes many developers can do both, but its difficult to do both at the same time & even harder to hire for.
overcast · 9 years ago
Here's my experience as a solo developer for all of my projects. When I have to do everything from marketing, SEO, domain/dns, server setup, to database design, interface design, and whatever else. Doing most of the actual programming in one language, rather than many, helps considerably in my workflow. The NPM ecosystem is also awesome.
Pr3fix · 9 years ago
Bingo. Having one language for your front end, back end, build tools, database schema... It simplifies life a lot.

You can reuse code / modules.

You Don't suffer from context switching inherent with jumping between languages.

Also, people act like JavaScript is a bad language purely because it is not strictly typed. It's an incredibly fast and robust language with a vibrant ecosystem -- something I'd argue is more important than pure technical bullet points.

ReverseCold · 9 years ago
Node.js let's you require() packages from npm. There, all the code you for ever need.

Instead of writing "a quick Python script" for personal use, I now write "a quick node script" because there's almost no thinking required on my part. Just require the packages and throw them together.

Not even Python has this package-completeness.

imagist · 9 years ago
Integrating a library requires a lot of thinking, or you'll integrate libraries that are immature and break, or don't exactly fit what you wanted and have to be hacked around. I've had to work in projects that used your methodology long-term and I would rather turn down a job than suffer through that again.
RUG3Y · 9 years ago
What do you mean by "package-completeness"?

While npm has a lot of packages, Python has a lot of fully matured, well-tested and well-documented packages.

It's apparent to me that there are a lot of low quality, duplicate and otherwise unstable npm packages.

Don't get me wrong, I actually like npm too but I don't think the Python comparison is even in the same ballpark.

DCoder · 9 years ago
The term for this is "Magpie-driven development".
zMiller · 9 years ago
Depends who you ask really.

If you are a developer then you should invest your time and be able to learn and use the best tool for the job.

If however you are something like a Product manager then learning JS is much more valuable time investment. It is the one language that will, currently, run anywhere ... Backend, Frontend, Side end, whatever.

You can quickly get good at JS to the point of being able to prototype the full app stack and come up with something that works, goto market and fail. Should you find your self in the unlikely position that your product meets its success, then you can go ahead and hire a Go/Elxir/WebASM whatever guy to optimize this and needs larger throughput.

You have 24 hours in day, 6 to sleep, 3 to breath, 1 to eat. How do you want to spend the remaining 14 ? You get good at what you practice :)

Deleted Comment

vkjv · 9 years ago
IMHO, the core piece is that supporting a language is very, very expensive: tooling, CI, linting--especially standards. Many of things that ensure quality are not language agnostic. It's not about learning a language, it's about maintaining one.

If you are a web shop you have javascript. Using node.js allows you to eliminate one extra language.

Is it perfect? Definitely not. Is it best for everything? Probably not the best for anything. But, it's good enough for most stuff. It doesn't make sense to incur the burden of another language for a one off task.

dominotw · 9 years ago
>"more people know JavaScript so you don't have to learn a new language, which saves you time".

This is also an excuse for NIH ( never invent here ) where people try to shoehorn an OSS product even if it doesn't fit their needs .

imagist · 9 years ago
I know Javascript well enough to avoid it when possible.
matthewaveryusa · 9 years ago
ECMAScript6 is a wonderful language that approaches the syntactic readability of python. If you're stuck with ECMAScript5 sure, go ahead and say javascript is terrible because it is. But ECMAScript6? No way. They fixed so many annoying things. the fat arrow that captures "this", proper classes and inheritance, maps and sets and string template literals are the biggest ones that jump to mind. es6 is to javascript like what c++11 is to c++ -- it made it so you don't have to use the warts. ECMAScript6 snippet:

  class Shape {
      constructor (id, x, y) {
          this.id = id
          this.move(x, y)
      }

      move (x, y) {
          this.x = x
          this.y = y
      }

      fixed_offset(off_x,off_y) {
        return () => {
          console.log(`offsets x: ${off_x} y: ${off_y}`)
          return [this.x + off_x, this.y + off_y]
        };
      }
  }
And yes ECMAScript5 is gross:

  var Shape = function (id, x, y) {
      this.id = id;
      this.move(x, y);
  };
  Shape.prototype.move = function (x, y) {
      this.x = x;
      this.y = y;
  };
  Shape.prototype.fixed_offset(off_x,off_y) {
    var self = this;
    return function() {
      console.log("offsets x: " + off_x + " y: " + off_y`)
      return [self.x + off_x, self.y + off_y]
    }
  }

smokeyj · 9 years ago
> As node.js is not multi-threaded, we spin up 4 instances of node.js per server, 1 instance per CPU core. Thus, we cache in-memory 4 times per server.

And why not use a shared memory server?

> Operations started adding rules with 100,000s of domains, which caused a single set of rules to be about 10mb large ... If we weren’t using node.js, we could cut this bandwidth by 4 as there would only be one connection to the Redis cluster retrieving rule sets.

Maybe a 10mb json string isn't the best design decision.....

Or you know, you could have one node process connect to the Redis server, and have the local processes read from a shared memory server.. Or you could not store your rules as a 10mb friggin JSON string..

> When rule sets were 10mb JSON strings, each node.js process would need to JSON.parse() the string every 30 seconds. We found that this actually blocked the event loop quite drastically

Well then do it in another thread and save it to shared memory. Maybe, just maybe, JSON strings aren't the tool for the job here.

ionrock · 9 years ago
While I don't know node.js very well and can't speak to the "shared memory server", I do think it is valid to point out that this sort of redesign to fit the platform is frustrating. I've hit similar situations in Python where an async i/o framework is used and at some point there becomes a CPU bound task. The application was redesigned and new microservices were started in order to cope and the architecture became convoluted. This sort of question always seems to always show up in Python where limitations of the GIL influences the design of the software, the libraries used and deployment. It seems node.js, not surprisingly, has the same problems.

As the above comment clearly points out, there are workarounds to make things work. Still, when I think about the amount of time and effort that goes into debugging systems when they hit limits like the author mentioned, I imagine most of the the benefits gained by using a language like Python or JavaScript go away.

gaastonsr · 9 years ago
It's not a Node.js limitation either. The design was not thought out from the beginning.

It has happened to me before. I design something, then I run it in multiple CPUS and it doesn't make sense anymore...

Or realize that if try to scale the current design doesn't hold.

Node.js running in a single thread is a good thing if you know how to work with it. Or if you really need multiple threads, use another PL.

Or you could write it in C++ and bind it to Node.js. And run it as a background process. Or spin up a worker? there seems to be multiple solutions here...

leopoldfreeman · 9 years ago
NodeJS is a multi-threaded process. You can verify it with top or ps command. Async methods to decode JSON: https://github.com/nodejs/node-v0.x-archive/issues/7543
tayo42 · 9 years ago
The thread thing in nodejs seems very misunderstood. Only the javascript runs in one thread. I think its libuv that uses 4 threads to do most of the work in node.
ec109685 · 9 years ago
Is that issue actually resolved?
rodp · 9 years ago
While I agree Node.js isn't the right tool for any job -- just like anything else, really -- after reading his description of the problem, I can't shake off this feeling that the main issues he has with performance in this case have very little to do with Node itself. Parsing a huge JSON string in any language would block CPU for a while. This JSON then becomes a huge hash table in memory, so no wonder each process uses up a lot of RAM. I don't know how these rules are then used but it seems to me he might be better off trying to rethink how to do shared memory in this case before he simply blames Node for blocking CPU and wasting memory.

That said, I can imagine other languages (like Java or Go) could still end up being more efficient than Node.

jonas21 · 9 years ago
The issue isn't that it takes a long time to parse the JSON. It's that the server can't do anything else while it's parsing. In Java, for example, you could parse the JSON on a background thread without affecting your ability to serve requests.

Similarly, the memory issue isn't so much that a single copy of the table takes a lot of space, but rather that they need to store 4 copies of the table -- because they're running 4 different processes in order to utilize multiple cores.

Both of these issues are specific to nodejs.

tyingq · 9 years ago
"Operations started adding rules with 100,000s of domains, which caused a single set of rules to be about 10mb large"

There's not enough detail to be sure, but this sounds more like "when a relational database would be a better idea than redis."

Edit: That is, pushing the evaluation of the rules down...rather than pulling a kv and walking 10MB (of JSON?) to get to the small number of rules that apply for the transaction.

binocarlos · 9 years ago
This is an excellent article which really highlights the underlying trade-offs when you choose node for your service (i/o bound work vs cpu).

Unless you know for sure what limits you will hit - it makes sense to iterate quickly and find out. Then, if the service is actually hitting limits (and probably not the ones you thought) - re-write it in a multi-threaded concurrent language like go, elixr etc - or a language designed to solve the actual problems the service is hitting (which might be disk i/o or other infrastructure level things not language choice)

dlojudice · 9 years ago
They could have fixed part of the architecture by having a "cache service" process (4 cpus: 3 for proxies, 1 for the cache service). With that they'd have a single point consuming their limited resources (memory, cpu and socket for redis connections), using IPC to communicate between process.
gaastonsr · 9 years ago
I thought the same or even the same 4 cpus for proxies and a shared 1 for the cache service.
neebz · 9 years ago
JSON.parse() is one issue we faced regularly. Any large amount of data fetching could block the event loop and the whole server slows down. It's very unforgiving.

We go great length to figure out which attributes to fetch and add limits to all our sql queries. These are best practices but with node they are must.

reaktivo · 9 years ago
There's always streaming json parsers[1] that will help you when parsing large json data sets.

[1]: https://github.com/dominictarr/JSONStream

beejiu · 9 years ago
Node.js isn't great for CPU bound tasks in general.
danenania · 9 years ago
Parsing json is sneaky because it can show up in apps that are otherwise IO bound (where Node shines) and don't seem like they should be CPU intensive on the surface.
wehadfun · 9 years ago
JavaScript is my first non-mathematical programming language and I haven’t found the need to expand my programming skills to more

-Having a hard time taking anything this guy says seriously