I didn't like the idea. I prefer the alternative approach: _I_ decide the order of dirs in the PATH env. If I introduce an executable with a name, that overrides a system one - I probably do that intentionally.
If I introduce an alias (like `grep='grep --binary-files=without-match --ignore-case --color=auto`) that matches the name of a system binary - I probably do that intentionally.
And if I EVER need to call grep without my alias - I just prefix it with a backslash: \grep will search with case sensitivity and no color and will scan binaries.
Looked so backwards to me, too. However, I decided to give it a go, anyway. Now, I have some scripts and small commands which start with a comma, and it looks neat and time saving.
Yes, I can do path ordering to override usual commands. However, having a set of odd-job scripts which start with a comma gives a nice namespacing capability alongside a well narrowed-down tab-completion experience.
While it's not the neatest thing around, it works surprisingly well.
Another idea which looks useless until you start using is text expanders (i.e.: Espanso and TextExpander).
Either adding your script directory in front of the PATH, or creating `alias` that provide a full path to your script where a conflict exists, makes a whole lot more sense to me.
I've never had this collision problem yet, despite appending my script directory to the end, but I'll use either of the above solutions if that ever becomes a problem.
One rarely actually needs to shadow binaries. Some cases could indeed be covered by introducing an alias that binds the binary's name to call a different copy of that binary.
You use shadowing to fix issues where you install some software that expects you to have a sane and ~recent version of some tool like git, but you don't as your system provides that binary and unfortunately it is either not sane (not GENERALLY sane [while it could be sane for system scripts]) or not recent enough. In that case the program's function would simply fail if it would call the system's binary and you shadow the binary with your version to fix that.
> adding your script directory in front of the PATH
That's a poor advice for the scripts you call relatively frequently. Instead, (as a general approach, we aren't discussing some particular script) don't use shadowing for scripts: just pick a non-conflicting script name and append the script's dir to $PATH.
Word of warning, I use `\grep` quite frequently. The usage is when you are piping after grep or saving to a variable.
Illustrative example:
$ TO_DL=$(curl "https://foo.com/releases/" \
| grep -e "latest" \
| head -n1 \
)
$ curl $TO_DL
curl: (3) bad range in URL position XX
https://foo.com/releases/latest.tar.gz
^^^^^^
Annoyingly `--color=auto` can change the representation of the characters so when you run again there's a mismatch. I just find `\grep` easier than `grep --color=never`.
Annoying footgun, but something to be aware of in case you go down this extremely confusing rabbit hole like me. I couldn't figure it out until I decided to hexdump the string.
[Side note]: My first thought reading the article was also about how `\` basically solves the problem but they do have one advantage in that they can do `,<tab>` and get a list of all their custom commands. Personally not worth it for me but I can definitely see this being useful for some. Especially on shared systems. But getting everyone to prefix a script name seems just as unlikely as getting everyone to place programs in the right location.
When “I” means me then this usually works for me. But when “I” becomes “we”, sometimes this goes off the rails because someone introduces a bin with breaking changes that silently fucks up projects that dev doesn’t really know about, or forgot about.
I would recommend against overriding standard system binaries, you could break compatibility on your system with scripts that depend on those binaries. I just use an abbreviation like rg=“grep -RE”
I do this, and routinely shadow commands with my own wrappers to do things like set environment variables.
And then there’s Claude. It deletes whatever it finds at ~/.local/bin/claude, so I have to use a shell function instead to invoke the full path to my wrapper.
Random flags added to core tools are done with aliases, which do not affect the launched processes, not by shadowing them in ~/bin. Shadowing in ~/bin are for cases where a newer (compared to the system-wide version) or custom version of a tool is needed.
Not really, since if one usually does that - they probably understand the possible consequences and don't shadow whatever they like, but do it carefully.
On MacOS I shadow that way just curl and git binaries to the versions installed from homebrew and nothing has broken (yet). I know that tar on MacOS is also a weirdo that I'd rather shadow with the homebrew's gtar, but their args are different and I of course understand that there's a high probability of something in system to be bound to mac's version of tar, so here I better remember to use 'sane' tar as gtar or use an alias (instead of shadowing the binary) for tar to use gtar (because aliases are for users, not for system scripts/processes).
And on my home desktop's Debian - I don't even use shadowing of binaries at all (never needed it).
Also, I just realized: I change PATH env via my shell's rc script (~/.zshrc), so I probably could worry even less about shadowing system binaries (like tar on MacOS) possibly breaking things.
Just on your first suggestion, this also means that if a person or process can drop a file (unknown to you) into your ~/bin/ then they can wreak havoc. Eg they can override `sudo` to capture your password, or override `rm` to send your files somewhere interesting, and so on.
Btw on the second suggestion, I think there's a command named `command` that can help with that sort of thing, avoids recursive pitfalls.
That would require someone to already want to sabotage me in particular, learn my private workflows, and also have write access to my home folder. At that point, All is Lost.
Don't tell people to sacrifice agency for apocalypse insurance that doesn't work, lol
Others have already given valid answers: grep is not ripgrep [their params don't match], so it's a bad idea to alias 'grep' to use ripgrep. But it's okay to alias 'ripgrep' (or 'rg' or whatever) to use ripgrep with some args.
Tangentially related. Don't ever put "." in your PATH. I used to do this to avoid typing the "./" to execute something in my current directory. BAD IDEA. It can turn a typo into a fork bomb. I took down a production server trying to save typing two characters.
Why would this own a server? ls lists itself, but listing itself shouldn't cause it to run again? Where's the infinite loop that brings the server down?
I like to follow my own convention where I name files with shell scripts with an extension: .sh for POSIX-compatible scripts, .bash for scripts with bashisms or .zsh for scripts with zshisms.
If I ever wanted to achieve what you initially wanted to achieve - I could use something like
alias -s sh=sh
alias -s bash=bash
alias -s zsh=zsh
Just like I do bind .txt and .conf to 'less', .pdf to 'qpdf', .json to 'ijq', video formats to 'mpv' and so on.
Somebody mentioned it elsewhere, but it is a security risk: if you end up in a directory that's not under your control, and you do a "ls", it might execute "./ls" instead of /usr/bin/ls, and that can be doing anything, including piping your ~/.ssh/id_* to a remote server.
This can also happen by downloading something off the internet (git clone, or tar xz foo.tar.gz), or on a multi-user system (eg. someone can put any of these common commands into /tmp/ and wait for you to drop into it and try a "ls" there) — if you have any untrusted content anywhere, you are exposed.
A trip down the recursion hole. Also, scripts will inherit the relative path so they will have different absolute paths from each other. Seems easier to just type ./ so it's kinda funny in a "UNIX haters handbook" kind of way, but it's not even a fault in linux's command interface in that case. We've all been there.
Oh, that's without even going into the security risks and loss of portability.
Presumably a script that aliases a common thing or something and then it uses the same. E.g. someone adds ./sed that has some default params and calls sed. You’re intended to call it with ~/not-in-path/defaulted/sed and it is supposed to then call sed but instead calls itself if it’s earlier in the path hierarchy.
Might even be as simple as “detect if I’m running gnu sed or bsd sed and use the appropriate one”. Obviously you can not have this problem by being smart about other things but defense in depth right?
Glad it worked for OP, but I've never once in 30+ years of this had a conflict that did something I didn't want. ~/bin/ is early in my PATH, and for a good reason. Things I put in there I want to take precedence, so I use this to purposely override provided bins. (Though I can only think of one time I wanted to do that, too.)
Kinda makes no sense to me: so you don't use '--' as a prefix, you use it in the middle of an alias, so you first have to autocomplete, say, 'gi' not to 'git' but to 'git--progress'. What does that alias do? Doesn't it call git with some args? If so - why not just alias it to git?
I actually have the below clashes function in my bashrc to detect overloaded names, and a few compgen aliases for sorted formatted command lists. Clashes takes a few seconds to chew through the 3000ish commands on my Steam Deck.
I also discovered the comma thing myself, and use it for `xdg-open <web-search-url>` aliases, i.e. `,aw xdg` searches archwiki for "xdg", since the SteamOS on this thing doesn't have manpages.
alias words='iftty xargs' # join by spaces
alias c='compgen -c | sort -u | words' # eg: c | grep ^k | chop
# ... a for aliases, f for functions etc ...
alias man='g !archman' # https://man.archlinux.org
alias ,aw='g !aw' # https://wiki.archlinux.org
alias ,mdn='g !mdn' # https://developer.mozilla.org
alias ,nix='g !nixpkgs' # https://search.nixos.org
# ... a dozen others ...
g() { local IFS=+; xdg-open "https://ddg.gg/?q=$*"; }
iftty() { if [ -t 1 ]; then "$@"; else cat; fi; }
chop() { # [TABS] [STARTCOL] [CUTOPTS..] # eg. ls ~/.co* | chop
set -- "${1:-2}" "${2:-1}" "${@:3}"
cut -c "$2"-$(($2+8*$1-2)) "${@:3}" | column #| less
}
clashes() { # [CMDS..] # print overloads eg: clashes $(c)
(($#)) || set -- $(compgen -A alias -A function)
local ty cmd; for cmd; do ty=($(type -at "$cmd"))
((${#ty[@]}>1)) && echo -n "$cmd "; done; echo
}
Prefixing commands solves the namespace problem and discoverability (at least partly). I use a slightly more sophisticated method, which helps me remember which custom utilities are available and how to use them: sd [1], a light wrapper written for zsh that, in addition to namespaces, provides autocompletion, custom help texts + some other QoL enhancements. Can definitely recommend if you're looking for something a bit more fancy.
I do something similar but with a project prefix. I maintain a lot of packaging scripts and settled on prefixing them by project - gps-build, gps-deploy, gps-sync etc. Tab completion does the rest.
The real issue isn't collisions though, it's discoverability. Six months later you forget half of what you wrote. The prefix at least lets you type gps-<tab> and see everything at a glance.
Worth pointing out that with Nix/NixOS this problem doesn't exist.
The problem in other distros is that if you prefix PATH so that it contains your executable "foo", and then run a program that invokes "foo" from PATH and expects it to do something else, the program breaks.
With Nix, this problem does not exist because all installed programs invoke all other programs not via PATH but via full absolute paths starting with /nix/store/HASH...
NixOS simultaneously smooths the path to using absolute paths while putting some (admittedly minor) speed-bumps in the way when avoiding them. If you package something up that uses relative paths it will probably break for someone else relatively quickly.
What that means is that you end up with a system in which absolute paths are used almost everywhere.
This is why the killer feature of NixOS isn't that you can configure things from a central place; RedHat had a tool to do that at least 25 years ago; it's that since most of /etc/ is read-only, you must configure everything from a central place, which has two important effects:
1. The tool for configuring things in a central place can be much simplified since it doesn't have to worry about people changing things out from under it
2. Any time someone runs into something that is painful with the tool for configuring things in a central place, they have to improve the tool (or abandon NixOS).
If it's a one off, you just use something like "nix shell" to add it to your path for running the script.
For non one-off sorts of things, you would substitute in the nix expression "${gnugrep}/bin/grep" the "${gnugrep}" will expand to "/nix/store/grep-hash" and also make a dependency on the gnugrep package, so that the grep install won't get garbage-collected as long as your package is still around.
Here's an example[1] from a package expression for e-mail client I use, which will shell out to base64 and file. Upstream relies on these two programs being in $PATH, but this replaces the string used for shelling out with the absolute path in the nix store.
For shell scripts, I'll just do something like this near the top:
GREP="${GNU_GREP:-$(command -v grep)}"
Then I use "$GREP" in the script itself, and develop with grep in my path, but it's trivial to prepend all of my dependencies when I bundle it up for nix.
[user@nixos:~]$ which grep
/run/current-system/sw/bin/grep
[user@nixos:~]$ ls -l /run/current-system/sw/bin/grep
lrwxrwxrwx 1 root root 65 Jan 1 1970 /run/current-system/sw/bin/grep -> /nix/store/737jwbhw8ji13x9s88z3wpp8pxaqla92-gnugrep-3.12/bin/grep
Basically, it is still in your environment, so I don't see how he can claim that this problem doesn't exist in Nix, unless you use flakes like a proper Nix afficionado.
Readit News
If I introduce an alias (like `grep='grep --binary-files=without-match --ignore-case --color=auto`) that matches the name of a system binary - I probably do that intentionally.
And if I EVER need to call grep without my alias - I just prefix it with a backslash: \grep will search with case sensitivity and no color and will scan binaries.
Yes, I can do path ordering to override usual commands. However, having a set of odd-job scripts which start with a comma gives a nice namespacing capability alongside a well narrowed-down tab-completion experience.
While it's not the neatest thing around, it works surprisingly well.
Another idea which looks useless until you start using is text expanders (i.e.: Espanso and TextExpander).
I've never had this collision problem yet, despite appending my script directory to the end, but I'll use either of the above solutions if that ever becomes a problem.
You use shadowing to fix issues where you install some software that expects you to have a sane and ~recent version of some tool like git, but you don't as your system provides that binary and unfortunately it is either not sane (not GENERALLY sane [while it could be sane for system scripts]) or not recent enough. In that case the program's function would simply fail if it would call the system's binary and you shadow the binary with your version to fix that.
> adding your script directory in front of the PATH
That's a poor advice for the scripts you call relatively frequently. Instead, (as a general approach, we aren't discussing some particular script) don't use shadowing for scripts: just pick a non-conflicting script name and append the script's dir to $PATH.
Thanks, mathfailure - this genuinely improves my life!
Word of warning, I use `\grep` quite frequently. The usage is when you are piping after grep or saving to a variable.
Illustrative example:
Annoyingly `--color=auto` can change the representation of the characters so when you run again there's a mismatch. I just find `\grep` easier than `grep --color=never`.Annoying footgun, but something to be aware of in case you go down this extremely confusing rabbit hole like me. I couldn't figure it out until I decided to hexdump the string.
[Side note]: My first thought reading the article was also about how `\` basically solves the problem but they do have one advantage in that they can do `,<tab>` and get a list of all their custom commands. Personally not worth it for me but I can definitely see this being useful for some. Especially on shared systems. But getting everyone to prefix a script name seems just as unlikely as getting everyone to place programs in the right location.
Call it the Chesterton’s Fence of ‘which’.
And then there’s Claude. It deletes whatever it finds at ~/.local/bin/claude, so I have to use a shell function instead to invoke the full path to my wrapper.
Because I cannot imagine much 3rd party scripts working with random flags added to core tools
Random flags added to core tools are done with aliases, which do not affect the launched processes, not by shadowing them in ~/bin. Shadowing in ~/bin are for cases where a newer (compared to the system-wide version) or custom version of a tool is needed.
On MacOS I shadow that way just curl and git binaries to the versions installed from homebrew and nothing has broken (yet). I know that tar on MacOS is also a weirdo that I'd rather shadow with the homebrew's gtar, but their args are different and I of course understand that there's a high probability of something in system to be bound to mac's version of tar, so here I better remember to use 'sane' tar as gtar or use an alias (instead of shadowing the binary) for tar to use gtar (because aliases are for users, not for system scripts/processes).
And on my home desktop's Debian - I don't even use shadowing of binaries at all (never needed it).
Also, I just realized: I change PATH env via my shell's rc script (~/.zshrc), so I probably could worry even less about shadowing system binaries (like tar on MacOS) possibly breaking things.
Btw on the second suggestion, I think there's a command named `command` that can help with that sort of thing, avoids recursive pitfalls.
Don't tell people to sacrifice agency for apocalypse insurance that doesn't work, lol
... and breaks existing scripts that reference the system one, right?
Deleted Comment
Deleted Comment
If I ever wanted to achieve what you initially wanted to achieve - I could use something like
alias -s sh=sh
alias -s bash=bash
alias -s zsh=zsh
Just like I do bind .txt and .conf to 'less', .pdf to 'qpdf', .json to 'ijq', video formats to 'mpv' and so on.
This can also happen by downloading something off the internet (git clone, or tar xz foo.tar.gz), or on a multi-user system (eg. someone can put any of these common commands into /tmp/ and wait for you to drop into it and try a "ls" there) — if you have any untrusted content anywhere, you are exposed.
Oh, that's without even going into the security risks and loss of portability.
Might even be as simple as “detect if I’m running gnu sed or bsd sed and use the appropriate one”. Obviously you can not have this problem by being smart about other things but defense in depth right?
Easy autocomplete, I know there won't be any collision, and which command is mine.
I also discovered the comma thing myself, and use it for `xdg-open <web-search-url>` aliases, i.e. `,aw xdg` searches archwiki for "xdg", since the SteamOS on this thing doesn't have manpages.
[1] https://github.com/ianthehenry/sd
The real issue isn't collisions though, it's discoverability. Six months later you forget half of what you wrote. The prefix at least lets you type gps-<tab> and see everything at a glance.
The problem in other distros is that if you prefix PATH so that it contains your executable "foo", and then run a program that invokes "foo" from PATH and expects it to do something else, the program breaks.
With Nix, this problem does not exist because all installed programs invoke all other programs not via PATH but via full absolute paths starting with /nix/store/HASH...
NixOS simultaneously smooths the path to using absolute paths while putting some (admittedly minor) speed-bumps in the way when avoiding them. If you package something up that uses relative paths it will probably break for someone else relatively quickly.
What that means is that you end up with a system in which absolute paths are used almost everywhere.
This is why the killer feature of NixOS isn't that you can configure things from a central place; RedHat had a tool to do that at least 25 years ago; it's that since most of /etc/ is read-only, you must configure everything from a central place, which has two important effects:
1. The tool for configuring things in a central place can be much simplified since it doesn't have to worry about people changing things out from under it
2. Any time someone runs into something that is painful with the tool for configuring things in a central place, they have to improve the tool (or abandon NixOS).
/nix/store/grep-hash -flags files | /nix/store/head-hash
instead of: "grep -flags files | head"?
For non one-off sorts of things, you would substitute in the nix expression "${gnugrep}/bin/grep" the "${gnugrep}" will expand to "/nix/store/grep-hash" and also make a dependency on the gnugrep package, so that the grep install won't get garbage-collected as long as your package is still around.
Here's an example[1] from a package expression for e-mail client I use, which will shell out to base64 and file. Upstream relies on these two programs being in $PATH, but this replaces the string used for shelling out with the absolute path in the nix store.
For shell scripts, I'll just do something like this near the top:
Then I use "$GREP" in the script itself, and develop with grep in my path, but it's trivial to prepend all of my dependencies when I bundle it up for nix.1: https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/no...