I had this conversation with a person on reddit just the other day. After I pointed that there is no difference between piping a script directly into bash and downloading a package and installing it, they told me that they can audit a script/package they download and that they have personally audited every bit of code on their linux box. The kernel, gnome and metasploit.
I got to thinking what it would take to do such a task, and mainly what I came up with was a whole damned lot of time. Assuming a person can read, understand and remember 5000 lines of code per day (which honestly I think is way more than is realistic), it'd only take 8.5 years to audit the linux kernel. Add in all the other stuff and I figure something on the order of two decades. And in the end you'd be running two decade old software or you'd have to start over.
At the end of the day, security comes from the personal and corporate economics of reputation, profit, and prison avoidance. Do your best to get your stuff from people who you judge to be trustworthy and rely on their own self interest to not be malicious and to do their best to protect their repositories. And rely on others to be good citizens and report the bad shit that happens to them so that things can be cleaned up.
Now I'm not saying to throw out security best practices, but people should be aware that the quality of their systems are built on trust in human nature and self interest.
The body of the shell script should be wrapped in a function which is invoked at the end. If you do not trust them to get this incredibly basic thing right, how can you possibly trust any of the other code they have written?
There are better strategies than blindly relying on third party reputation. A primary approach is package management with hash databases, to ensure you are receiving the same code as others. Another is the use of multiple implementations whose results are compared, recently discussed @ https://news.ycombinator.com/item?id=12666923
You're still relying on the source to not be underhanded and malicious. Getting the same, reproducible build as everyone else only means you've got a lot of company. It doesn't mean there isn't malicious code or intentional "bugs."
> there is no difference between piping a script directly into bash and downloading a package and installing it
Well then I guess you haven't seen this, in which the server detects whether the script is being downloaded to disk or piped to a shell, and adjusts the payload accordingly:
People who pipe it to a shell aren't going to look at it anyway. That's kind of the point. If they were, they'd download it first. There is literally no security gained by downloading it and running it, instead of just `curl|sh`. IF it's malicious, you just installed it either way. The only thing downloading does in this scenario is lets you look at the script later on when you realize it might have been malicious, but that's even assuming you kept it (I would wager that almost everybody would throw away the script after installing it if they weren't planning on reading it before installation), and assuming you know enough to understand its contents (not everybody knows bash scripting, and you can write some surprisingly complex scripts).
I'm really not a fan of this author's style. The content seems angry just for the sake of it. I'm no expert on it, but it's pretty unclear whether the complaint is even valid, as there are talented, non-"retarded" developers that disagree:
There are ways of communicating beyond those designed to gain fake internet points on a west coast venture capital firm's website. The internet isn't your safe space. Anger is a gift.
`curl | sh` is insecure if you're using an http url, although as has already been said here, it's not really any more insecure than "download this script and run it", unless you're expecting all of your users to actually read through the whole script first. But if you're using an https url then you should be ok since the page cannot be hijacked or modified en route (unless the attacker actually has a trusted certificate for the domain, which is an attack that's way out of scope of this discussion).
The biggest risk with this approach, which the page doesn't even mention, is the danger of the connection being terminated before the whole script is downloaded, as the shell will still evaluate what was sent. But this can be handled in the script by making sure that an early EOF means nothing is actually run (e.g. you can wrap the whole script in a bash function, and then the last line executes the function).
So if you're using an https url, and the script is written to be resilient against early termination, then this is a perfectly reasonable way to install things.
There's a few subtle differences you haven't mentioned:
1) If you attempt to read the script in your browser first, and everything's great, then go pipe to bash, the server can send alternate content based on your user agent.
2) You'd have a hard time proving the first point, or reviewing to see if a script acted poorly, if you didn't have a local copy, which piping to a shell like this normally prevents you from doing.
> If you attempt to read the script in your browser first, and everything's great, then go pipe to bash, the server can send alternate content based on your user agent.
If you can't trust the site to give you a safe installer then you can't trust the rest of the sources it gives you either--you would need to audit the entire package. Virtually nobody is going to do that. Singling out the installer as uniquely dangerous is security theater.
> 1) If you attempt to read the script in your browser first, and everything's great, then go pipe to bash, the server can send alternate content based on your user agent.
I'm having trouble finding a proper source, but I'm pretty sure someone came up with a way to do it even if curl spoofs the user agent. I think it worked by looking for the characteristic timing pattern of bash emptying the pipe in small bursts as it executes the individual script lines.
> If you attempt to read the script in your browser first, and everything's great, then go pipe to bash, the server can send alternate content based on your user agent.
curl | less
Or copy the request "as curl" from the network tab of any modern browser.
There was a post on HN a while ago where somebody wrote a server side tool which can detect curl -> bash piping and delivered different content if that happens.
Sure, but how is that any less secure than other installation alternatives? As the site owner, I could swap out a valid package for a troll package at any moment to any subset of users I wanted to if they're not validating the contents of it...
This is not a risk inherent to `curl | sh`, it's a risk that you run any time you download software from anywhere that doesn't have a signature using a key that you've already previously verified as trustworthy.
> if you're using an https url then you should be ok
You're trusting the server in this case, not the contents being provided by the server. There's no way to verify that the contents attributed to eridius on github are the same contents that eridius actually put up there.
Github in particular has been shown to have problems with spoofed users.
Let's go one step further. I have a `curl | sh` script for you to execute to install Docker. Just run:
curl -fsSL https://get.docker.com/ | sh
Oops. That was actually get.rekcod.com, with a RTL unicode character embedded. Or a unicode C which doesn't map to the ascii 'c'. Or I am able to mitm and strip the TLS. Or...
The vulnerabilities go on and on. Using only HTTPS, you have no way to prove that the code you got from get.docker.com is the code that the docker engineers intended for you to use.
"curl | bash" isn't actually very different security wise from "download this piece of software from our webpage and install it". Every time you install software these days you put a certain amount of trust on the vendor. (This may change in a theoretical future of reproducible builds and binary transparency logs.)
I'd argue that "curl http://.* | sh" is always bad, but so is every webpage offering software downloads that isn't https by default (and there are plenty of them).
How is this more of a security risk than having the user do wget https://example.com/whatever.deb and then dpkg -i whatever.deb? Or adding their apt repo & public keys? Sure, the project maintainer could include malicious code with a curl bash, but they could do in either of the ways I mentioned as well.
And maybe it's not relevant, but I find it really off-putting how the author calls these developers idiots and retards constantly.
It's definitely off-putting. And I'm totally fine with curl | bash. Composer (PHP package manager) does that and I've used it hundreds of times (production is containerized, but I do this on my personal machines too).
BUT. There is a difference -- code signing. HTTPS ensures that the data isn't compromised en route, trust in the vendor is what makes you OK with letting them run code on your computer, but neither of those things protect against a compromised payload. ie, if the vendor's server gets hacked and the script replaced, HTTPS doesn't help, and you get code that the vendor never intended for you to run. Code signing is what protects against this, cryptographically ensuring that the code you got is exactly the code the vendor wanted you to have, and is the last link in the chain that connects your machine to a trusted vendor.
But it also requires you to have some way to actually validate that the signature is valid. Apple provides this service to registered Apple developers, so you can code-sign your independently-distributed app or installer and the certificate you sign with is generated and signed by Apple. But in nearly all cases of code signing I've seen outside the Apple developer ecosystem, it's GPG signatures, which relies on you being able to independently verify that the signing key is valid and belongs to the vendor and was not compromised. Which is to say, not very many people who download stuff outside of a package system are actually going to validate that sort of thing.
Most people happily install whatever GPG key to apt the website tells them too. If they can compromise the installer probably they can compromise the page too.
Secondary fun fact when you add a key to GPG it's valid for any package from any repository. That repository could happily replace libc, or perhaps more strangely a key used to sign some popular repository could be uploaded to say the main Debian archive and anyone with that key added would happily install it with no errors.
I often see the claim that curl|bash is no worse than what a package manager does. That is simply untrue, for the following reasons.
(1) HTTP (no S) MITM. At least the lazy devs admit this one.
(2) No key/signature checking at all. Sure, some semi-lazy devs will tell you to add their own repo, and maybe you don't check the key for that repo yourself, but there are others who do and they'll raise an alarm you might hear. With curl|bash you don't even get this kind of herd immunity.
(3) No dependency checking.
(4) No adherence to standards. If you've ever tried to get a package included in e.g. Fedora or Debian, you know that there are people who will go over them with a fine-tooth comb and will reject them if they do bad things (or do them in a bad way).
(5) Most install scripts don't handle interrupted downloads well unless the author has taken special care (thank you for this one eridius). If you're piping directly into the shell you have no idea whether that's the case, and if the dev's lazy enough to be doing things this way in the first place the odds are poor.
Packages and package repos can be deployed and used in many ways. Some ways provide pretty strong safeguards and guarantees; other ways are weaker. Curl|bash is weaker than any of them. There's just no excuse.
Indeed, I think most of the harms of curl | bash fall under operational rather than security. To add to your list:
6) reproducibility: vendors could version the download links but I've not seen that done, so you're always getting the latest version which depending on what you're doing might not be what you want.
7) uninstall: maybe the vendor was nice enough to include an install script? Sure with deb/rpm a vendor can screw up the uninstall but the framework is there for them to do it right, with curl | bash the vendor needs to understand that's something they need to do and implement a solution on their own.
8) am I really running bash? And the version you expect me to be? Hopefully your distro isn't evil, but I have more than once found bash just a sym link to something less feature rich. Same goes with older versions, you'd be surprised what features are "new"
I also find it a bit odd because the cranky old sysadmin who solves everything with unintelligible shell scripts is often made a mockery of in this world of saltstack, docker, cloud and other buzzwords - yet now we're just signing up for an even worse version of it?
(1) you can easily see and just not use it
(2) is incorrect, see https://rvm.io -- it verifies the script
(3) you can do within the script, though you'd have to handle different package managers
(4) is true for packages as well. You can put whatever you'd like in an RPM or deb including e.g. A post-install hook to sudo rm -rf --no-preserve-root /
Making packages is hard if you want to support many distros. Maybe we should adopt the Go model and just ship binaries.
Underneath the curl|bash method, which is what is being discussed I believe, there is a link to a "more secure installation.": https://rvm.io/rvm/security . These instructions have you download and verify a signed installer.
This is a perfect example of dogma over logic. I'm not sure if the author just prefers we all use the Mac app store. Three rules for any installation process:
In the world of signed payloads the delivery, (2), includes all the distribution infrastructure. And trusting the vendor, 1, means verifying signature of the payload with their public keys. The assumption here being the building and signing infrastructure is more secure than the distribution infrastructure.
This level of security isn't desirable for everyone in all situations, but it's not just dogma and it's not "the same thing".
I don't disagree with the author that it is a huge security risk. But this is yet another example of a software enthusiast who doesn't get the value of convenience. In fact, "convenience" is probably the first attribute of great software.
So again, while I agree with their general sentiment, being "baffled by how [oh my zsh] became so popular" just because it instructs users to curl pipe shows that they don't get the core issue at play here.
Readit News
I got to thinking what it would take to do such a task, and mainly what I came up with was a whole damned lot of time. Assuming a person can read, understand and remember 5000 lines of code per day (which honestly I think is way more than is realistic), it'd only take 8.5 years to audit the linux kernel. Add in all the other stuff and I figure something on the order of two decades. And in the end you'd be running two decade old software or you'd have to start over.
At the end of the day, security comes from the personal and corporate economics of reputation, profit, and prison avoidance. Do your best to get your stuff from people who you judge to be trustworthy and rely on their own self interest to not be malicious and to do their best to protect their repositories. And rely on others to be good citizens and report the bad shit that happens to them so that things can be cleaned up.
Now I'm not saying to throw out security best practices, but people should be aware that the quality of their systems are built on trust in human nature and self interest.
Well then I guess you haven't seen this, in which the server detects whether the script is being downloaded to disk or piped to a shell, and adjusts the payload accordingly:
https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
i.e. when you download to disk and look at it, it looks benign, but when you pipe it to a shell, you get owned.
Just say no to curl|sh, kids.
curl >x cat x . x
However, the point is that if you can't trust the publisher it doesn't make any difference what the installation procedure is.
Deleted Comment
https://sandstorm.io/news/2015-09-24-is-curl-bash-insecure-p...
And for an egregious example of the author's approach of being angry without validating anything:
https://gnu.moe/petpeeves.html <- an angry post complaining about English mistakes that is itself splattered with basic grammar errors
The following on the home page [0] is hilarious:
[0] https://gnu.moe/`curl | sh` is insecure if you're using an http url, although as has already been said here, it's not really any more insecure than "download this script and run it", unless you're expecting all of your users to actually read through the whole script first. But if you're using an https url then you should be ok since the page cannot be hijacked or modified en route (unless the attacker actually has a trusted certificate for the domain, which is an attack that's way out of scope of this discussion).
The biggest risk with this approach, which the page doesn't even mention, is the danger of the connection being terminated before the whole script is downloaded, as the shell will still evaluate what was sent. But this can be handled in the script by making sure that an early EOF means nothing is actually run (e.g. you can wrap the whole script in a bash function, and then the last line executes the function).
So if you're using an https url, and the script is written to be resilient against early termination, then this is a perfectly reasonable way to install things.
1) If you attempt to read the script in your browser first, and everything's great, then go pipe to bash, the server can send alternate content based on your user agent.
2) You'd have a hard time proving the first point, or reviewing to see if a script acted poorly, if you didn't have a local copy, which piping to a shell like this normally prevents you from doing.
If you can't trust the site to give you a safe installer then you can't trust the rest of the sources it gives you either--you would need to audit the entire package. Virtually nobody is going to do that. Singling out the installer as uniquely dangerous is security theater.
I'm having trouble finding a proper source, but I'm pretty sure someone came up with a way to do it even if curl spoofs the user agent. I think it worked by looking for the characteristic timing pattern of bash emptying the pipe in small bursts as it executes the individual script lines.
curl | less
Or copy the request "as curl" from the network tab of any modern browser.
If you execute unsigned code from a 3rd party without sandboxing, transport encrytion doesn't help you much.
Similarly to what happened to e.g. https://transmissionbt.com/keydnap_qa/ or http://www.classicshell.net/forum/viewtopic.php?t=6441
You're trusting the server in this case, not the contents being provided by the server. There's no way to verify that the contents attributed to eridius on github are the same contents that eridius actually put up there.
Github in particular has been shown to have problems with spoofed users.
Let's go one step further. I have a `curl | sh` script for you to execute to install Docker. Just run:
Oops. That was actually get.rekcod.com, with a RTL unicode character embedded. Or a unicode C which doesn't map to the ascii 'c'. Or I am able to mitm and strip the TLS. Or...The vulnerabilities go on and on. Using only HTTPS, you have no way to prove that the code you got from get.docker.com is the code that the docker engineers intended for you to use.
I'd argue that "curl http://.* | sh" is always bad, but so is every webpage offering software downloads that isn't https by default (and there are plenty of them).
And maybe it's not relevant, but I find it really off-putting how the author calls these developers idiots and retards constantly.
BUT. There is a difference -- code signing. HTTPS ensures that the data isn't compromised en route, trust in the vendor is what makes you OK with letting them run code on your computer, but neither of those things protect against a compromised payload. ie, if the vendor's server gets hacked and the script replaced, HTTPS doesn't help, and you get code that the vendor never intended for you to run. Code signing is what protects against this, cryptographically ensuring that the code you got is exactly the code the vendor wanted you to have, and is the last link in the chain that connects your machine to a trusted vendor.
Secondary fun fact when you add a key to GPG it's valid for any package from any repository. That repository could happily replace libc, or perhaps more strangely a key used to sign some popular repository could be uploaded to say the main Debian archive and anyone with that key added would happily install it with no errors.
Plenty of side cases in reality
(1) HTTP (no S) MITM. At least the lazy devs admit this one.
(2) No key/signature checking at all. Sure, some semi-lazy devs will tell you to add their own repo, and maybe you don't check the key for that repo yourself, but there are others who do and they'll raise an alarm you might hear. With curl|bash you don't even get this kind of herd immunity.
(3) No dependency checking.
(4) No adherence to standards. If you've ever tried to get a package included in e.g. Fedora or Debian, you know that there are people who will go over them with a fine-tooth comb and will reject them if they do bad things (or do them in a bad way).
(5) Most install scripts don't handle interrupted downloads well unless the author has taken special care (thank you for this one eridius). If you're piping directly into the shell you have no idea whether that's the case, and if the dev's lazy enough to be doing things this way in the first place the odds are poor.
Packages and package repos can be deployed and used in many ways. Some ways provide pretty strong safeguards and guarantees; other ways are weaker. Curl|bash is weaker than any of them. There's just no excuse.
6) reproducibility: vendors could version the download links but I've not seen that done, so you're always getting the latest version which depending on what you're doing might not be what you want.
7) uninstall: maybe the vendor was nice enough to include an install script? Sure with deb/rpm a vendor can screw up the uninstall but the framework is there for them to do it right, with curl | bash the vendor needs to understand that's something they need to do and implement a solution on their own.
8) am I really running bash? And the version you expect me to be? Hopefully your distro isn't evil, but I have more than once found bash just a sym link to something less feature rich. Same goes with older versions, you'd be surprised what features are "new"
I also find it a bit odd because the cranky old sysadmin who solves everything with unintelligible shell scripts is often made a mockery of in this world of saltstack, docker, cloud and other buzzwords - yet now we're just signing up for an even worse version of it?
Making packages is hard if you want to support many distros. Maybe we should adopt the Go model and just ship binaries.
1. Make sure you trust the vendor
2. Make sure you trust the delivery (TLS)
3. Think twice before you sudo
This level of security isn't desirable for everyone in all situations, but it's not just dogma and it's not "the same thing".
So again, while I agree with their general sentiment, being "baffled by how [oh my zsh] became so popular" just because it instructs users to curl pipe shows that they don't get the core issue at play here.
"Never, ever underestimate the power of convenience."
Witness the results of autorun on Windows removable media and Javascript in PDFs.