* Respect the user's default foreground and background color. Don't change them without good reason.
* If you use colors, make them legible regardless of what the default background and foreground colors are, and regardless of the terminal's color map.
* Don't use color as the only indication of something. The user's terminal might not display it, and it probably won't be preserved in copy&paste into notes.
* Use emoji only judiciously, if at all. Similar with gratuitous non-ASCII characters. It doesn't display everywhere, it doesn't paste well everywhere, and emoji can be a bit much when copy&pasted into some notes.
* In a scrolling (non-full-screen) stdout-ish output, don't delete important information that you showed temporarily. For example, hiding warnings or filenames compiled, to display a green checkmark for done. For another example, clearing the screen of Web app build information (including package security warnings!), to display a message that it's running in dev mode, is also not wanted. People might want to see that information, or copy&paste it into notes.
* If you went full angry fruit salad with your command line program, because it's your baby, and you're having fun hamming it up, that's fine, but please provide an easy preference setting for people to opt out of that. Your program is probably only one of many things on user's workstation display, where other programs might be using color and visuals more meaningfully, so animated throbbing red explosions for the code reformatter is a bit much.
That reminds me of an incredibly annoying bug I encountered a few years ago involving Docker. One of the scripts being run was outputting emoji to STDOUT, and this was causing the interactive terminal to crash and thus the container to exit. (This issue [0] has error-strings and simple repro tests.)
I'm not sure if the root cause ever got fixed, but I (and many others) ended up making PRs for various open-source projects, to grudgingly implement workarounds that compromised their original artistic vision. :p
Nit: ┐ isn't emoji. It's a box drawing character, and they've been around in one form or another since at least the 80s. The modern form has been part of Unicode from (nearly?) the beginning.
In any case, the character itself is unrelated to the bug. As someone pointed out in the bug thread, certain Cyrillic strings will also cause it.
> * If you use colors, make them legible regardless of what the default background and foreground colors are, and regardless of the terminal's color map.
This is kind of impossible. One can potentially provide for customized colors, though.
You can delegate the choice to the terminal by using ANSI color codes [1]. Then the onus is on the user/terminal developer to make sure the colors they’ve configured (or the defaults provided) are reasonable.
A downside of this is that it is quite restrictive, there are only like 8 colors.
EDIT: I missed the “regardless of the color map” bit, that is a bit unreasonable. Either you trust the terminal emulator or don’t. I think trying to have it both ways is too much.
As the article explicitly stated, these are descriptive not prescriptive rules. They're things that you can generally assume all terminal programs already follow.
> Use emoji only judiciously, if at all. Similar with gratuitous non-ASCII characters.
This deserves to be highlighted more. Nowadays some authors keep assuming users use a patched font and build their tools with that in mind, thus hampering ease of use significantly.
30%? As far as I know, about 10% of male population has red-green deficiency, in females it's much rarer, and and all the other forms of colorblindness are very rare regardless of sex.
> Don't use color as the only indication of something. The user's terminal might not display it, and it probably won't be preserved in copy&paste into notes.
And some of us choose to disable color by default. (Yes, I'm old-fashioned.)
Nice writeup. Since she mentioned how hard it is to learn these conventions, I'll plug my preferred reference when thinking about CLIs specifically (rather than TUIs and REPLs) - the Command Line Interface Guidelines https://clig.dev
It does include the blog post's rules on exiting on Ctrl-C, accepting `-` for stdin, disabling color in pipes, and much more.
I would also recommend, call to attention, the Further Reading from CLIG, https://clig.dev/#further-reading. POSIX, GNU, Unix resources, Heroku CLI guide, and 12-factor CLI app guide.
> programs should print “regular” output to stdout and errors to stderr
This is really important. I'd like to expand on this.
Standard output is for the data the program was asked to produce, no more and no less. If user asked for some JSON data, standard output should contain that exact JSON object and absolutely nothing else.
Standard "error" is actually a misnomer. It should have been called the standard user stream. Anything meant for the user to read on the terminal is supposed to go there. Error messages are of course included in that set but so are status messages and verbose output.
This ensures the output of programs can be piped into other programs seamlessly. Non-output data still gets sent to the terminal or redirected somewhere else.
Would have been great if programs were able to easily create new terminal-connected file descriptors for specific purposes. They could document those numbers in their manuals just like they document exit codes. Then users would get "ports" for every output. Could cut down on parsing significantly.
For compatibility, they could all redirect to either standard output or standard error by default... I think I'm gonna experiment with this a bit.
This is definitely one of the things that PowerShell got right. 6 different streams, each of which can be intercepted separately and configured differently.
There's a lot of good ideas in power shell. I like how they broke up the log stream into separate streams by severity. It still seems arbitrary and insufficiently general though. Why severity and not some other criteria?
What if programs could define any number of output streams? By default they could all coalesce into the terminal but other programs could connect to each one separately if they needed. Like an audio mixer of sorts: by default you get the mixed audio but there are ways to access each individual voice if needed.
I don't think that 6 different streams that will always go to the same place but some of them are usually turned off are very useful. Standard-error/user is useful because it goes to a different place. Levels would be better suited to a simple global variable, or a level associated with each printed line. The chance that someone redirects errors to error.log is reasonable, but the chance that someone redirects errors to error.log and warnings to warning.log and debug to debug.log and verbose to verbose.log is pretty low.
Also related to the article, PowerShell has PSReadLine, which implements a pretty reasonable text editor, including selection, copy/paste, classic keybindings such as Home/End/Ctrl-arrows, sane multiline command editing, semantic autocomplete and custom actions that can operate on input AST.
Once you get used to having all that, going back to other shells is pretty hard.
What about "debug"? I think the important part about stderr is that it's "meta" info about how the program is running (or ran) rather than what the program is intended to produce as output. This seems pretty similar to what often is referred to as the "debug" level of logging (which includes lower levels of logging like warnings and errors).
- If this is your first time hearing about the readline/emacs keybindings like Ctrl-E and Ctrl-W, you'll be pleased to know that most macOS input sources use these keybindings. If you're on macOS, feel free to try Ctrl-E, Ctrl-W, or Ctrl-U in your browser's address bar right now
- If you're using a command line program that doesn't support _any_ line editing (e.g. no readline keybindings, and no other keybindings), you can install the `rlwrap` program and launch the REPL under rlwrap. For example Standard ML of New Jersey has a REPL but no line editing functionality, but you can recover that via `rlwrap smlnj`
- "don’t use more than 16 colours" — I would go so far as to say "don't use more than 8 colors, or at least make your colors configurable." Many popular color schemes, including Solaraized and the default Base 16 color scheme, use the "bright" colors to hold various shades of gray. What you think is "bright green" might actually be the same shade of gray that normal text is colored.
> If you're on macOS, feel free to try Ctrl-E, Ctrl-W, or Ctrl-U in your browser's address bar right now
Most browsers I've used close the current tab when you press Ctrl-W. Actually, the terminal emulator I use, Alacritty, also does this, and most file explorers that have tabs also do. Iirc even windows explorer does this now, but it's been a while since I've actually used windows.
>Actually, the terminal emulator I use, Alacritty, also does this
This can't be right. I use ctrl+w all the time, and occasionally use Alacritty. I'd notice if this shortcut closed my terminal window (it's extremely annoying when I use a web-based ssh, because I have this shortcut deep in my muscle memory).
Vim tells you how to quit when you press ctrl-c. It even tells you how to quit the moment you launch it. The meme about how impossible it is to quit Vim is nonsense.
No, this is in Emacs shell-mode on Linux (various distros over the years). Just tried a few other places: it works in qterminal, and also doesn't work in eshell. I've been writing Haskell for about 15 years, including my current job as a full-time Haskell developer, and I never realised GHCi supports Ctrl-D in some situations!
Every other REPL I've used handles this fine, e.g. for quitting Python, Nix repl, SSH sessions, and even the shell itself. Weird.
I mention it because while "yourprogram ~/path/to/file" always works, having a repl that asks for a filename might not work. I've seen a lot of software where this DOES work, so I think it counts as a "most TUI programs do this" thing.
Tilde expansion is the first operation in word expansion:
> The expansions that are performed for a given word shall be performed in the following order: 1) Tilde expansion, parameter expansion, command substitution, and arithmetic expansion shall be performed, beginning to end. 2) Field splitting shall be performed on the portions of the fields generated by step 1. 3) Pathname expansion shall be performed, unless set -f is in effect. 4) Quote removal, if performed, shall always be performed last.
To see the whole shell specification, goto the main page at https://pubs.opengroup.org/onlinepubs/9799919799/. Select "Shell & Utilities" from the top-left frame, then "Shell Command Language" from the bottom-left frame.
In my opinion it should remain a shell thing. Adding tilde expansion just complicates the implementation of the tool. Now you need to know the current users home directory.
And some tools might implement it incomplete so that '~' works but '~name' does not work.
In POSIX shells '~name' expands to the home directory of user 'name'
$ echo ~bin
/bin
Now your tool need a way to query home directory for any user including system users. Depending on NSS configuration this is more complicated than just reading /etc/passwd.
All for the rare case that someone passes a file path starting with tilde and wants it to be expanded. IMO, when you provide a file interactively you do so through a shell and tilde expansion is handled by the shell and otherwise just provide the actual file path and do not rely on the tool doing additional expansions.
PS: I mixed up TUI and CLI program a bit in my head. For an interactively used TUI program it might be beneficial to implement tilde expansion (but then it should be as complete as in Shells). A CLI program should not do magic stuff like tilde expansion.
I remember reading some rant about how Unix sucks because of the shell's filename expansion, because programs can't tell whether you wrote a star or actually just all the files in the directory. If programs could tell the difference, then you could get the desired renaming behavior from mv like this:
mv *.m *.c
They suggested that this filename expansion should be in the C standard library, not the shell.
Going further, as part of shell expansion, expanding the tilde could also be useful for filename expansion in the standard library.
I would not do that to your own REPL unless you are really serious about it, with good documentation, escaping, etc...
What if there is a file named "~"? What about "~username"? How do you escape/quote it? What if $HOME is not set, or set to something different than the actual user home directory? What about Windows? Also, a lesser known fact is that typing "~." after newline in ssh will force close the connection from the client side and there is nothing the server can do about it, so don't make it part of your workflow.
It's even worse than that, there are others, much more likely key combinations that are interpreted by the SSH client. For example "~v" increases the verbosity of the SSH client, "~C" opens a command line, etc.
Entering "~vicky/doc.txt" or "~Ricky/doc.txt" over SSH will actually only write "icky/doc.txt", and entering "~Chris/doc.txt" will have SSH print out it's internal command-line's usage instructions in the middle of the TUI. Screenshot from Vim: https://imgur.com/a/NCt0G9r
Oh, and I just remembered where this always burns me. It's cases like this:
./myprogram --config=~/.config/myprogram
Of course, many flags parsers are aware of this, or at least accidentally aware of this, because you can write it in a form that your shell will expand.
./myprogram --config ~/.config/myprogram
For some reason, my muscle memory requires me to type the =. I don't know why. It's probably a habit I picked up from a former employer's flag parsing library. I also always use --underscores_like_this instead of --hyphens-like-normal-people for the same reason. Sigh!
While tab completion can be fixed in appropriate contexts using custom completion functions (e.g., [1]), AFAIK there's no way to customize tilde expansion contexts (other than the fact that tilde expanding variable-style arguments is disabled in POSIX mode).
This would massively complicate kernel path resolution on many systems. Taking Linux as an example, even in the simplest cases, ~user expansion as typically implemented — via the getpwnam(3) libc function, which in itself implies a rather unfortunate dependency — requires reading and parsing /etc/passwd, and may also require access to network directory services like LDAP.
The main reason I enjoy CLIs so much more than GUIs (or eves TUIs sometimes) is that it feels so consistent.
There are conventions, but following all the conventions in a CLI is a lot easier than designing a good GUI. So they tend to be higher quality as a result.
I spend a lot of time thinking how to bring this property to GUIs, but my best answers are still "lots of effort" or "lower your expectations".
A large part of this I think is that other than MacOS, GUI conventions aren’t heavily implemented by the OS and easy to opt into. And because no other OS landed on “meta key for GUI shortcuts” so there are a lot more conflicts (e.g. ctrl-c).
The Application Framework defaults which underly most native macOS application build a lot of common keyboard controls in. Use the default menu bar classes with the default basic commands, and Command O, N, Q, X, C,V and probably others come for free, you just implement the code that you need to for those functions. Use a standard text field and you automatically get Command/Option/Fn Left, Right, Up, Down for navigation. It’s more notabke when a macOS application doesn’t follow convention (e.g. InteliJ uses Shift-Opt up/down to move lines rather than expand the selection by paragraph) than when one does.
Windows does decently well on this front, but ctrl as a default modifier can hurt terminal based app usage and there are a number of UI frameworks even within the OS that appear to get different defaults.
And in the Linux world, I think the only way you could do this would be for someone to design (and a distro to standardize on and port apps to) a full on application framework. The window managers don’t want to be in the business of dictating the behavior of stuff in windows. The GUI toolkits don’t want to be in the business of defining os wide defaults and the DEs and distros don’t want to be in the business of if porting or dictating UI frameworks. And realistically there’s no one “on high” that could make the sort of dictation that for example “hence forth copy and paste will be Meta-C and Meta-V”
I agree, but this article is -- as the author tries to make clear -- descriptive, not prescriptive. She's listing out the things she's seen commonly in applications, not trying to convince applications that they should behave in certain ways.
Readit News
* Respect the user's default foreground and background color. Don't change them without good reason.
* If you use colors, make them legible regardless of what the default background and foreground colors are, and regardless of the terminal's color map.
* Don't use color as the only indication of something. The user's terminal might not display it, and it probably won't be preserved in copy&paste into notes.
* Use emoji only judiciously, if at all. Similar with gratuitous non-ASCII characters. It doesn't display everywhere, it doesn't paste well everywhere, and emoji can be a bit much when copy&pasted into some notes.
* In a scrolling (non-full-screen) stdout-ish output, don't delete important information that you showed temporarily. For example, hiding warnings or filenames compiled, to display a green checkmark for done. For another example, clearing the screen of Web app build information (including package security warnings!), to display a message that it's running in dev mode, is also not wanted. People might want to see that information, or copy&paste it into notes.
* If you went full angry fruit salad with your command line program, because it's your baby, and you're having fun hamming it up, that's fine, but please provide an easy preference setting for people to opt out of that. Your program is probably only one of many things on user's workstation display, where other programs might be using color and visuals more meaningfully, so animated throbbing red explosions for the code reformatter is a bit much.
That reminds me of an incredibly annoying bug I encountered a few years ago involving Docker. One of the scripts being run was outputting emoji to STDOUT, and this was causing the interactive terminal to crash and thus the container to exit. (This issue [0] has error-strings and simple repro tests.)
I'm not sure if the root cause ever got fixed, but I (and many others) ended up making PRs for various open-source projects, to grudgingly implement workarounds that compromised their original artistic vision. :p
[0] https://github.com/docker-archive/toolbox/issues/695
In any case, the character itself is unrelated to the bug. As someone pointed out in the bug thread, certain Cyrillic strings will also cause it.
This is kind of impossible. One can potentially provide for customized colors, though.
A downside of this is that it is quite restrictive, there are only like 8 colors.
EDIT: I missed the “regardless of the color map” bit, that is a bit unreasonable. Either you trust the terminal emulator or don’t. I think trying to have it both ways is too much.
[1]: https://gist.github.com/JBlond/2fea43a3049b38287e5e9cefc87b2...
There is some effort to standardize this: <https://bixense.com/clicolors/>
This deserves to be highlighted more. Nowadays some authors keep assuming users use a patched font and build their tools with that in mind, thus hampering ease of use significantly.
The user may be colour-blind (30% of the population). The user may be completely blind and relying on a screen reader.
And some of us choose to disable color by default. (Yes, I'm old-fashioned.)
Deleted Comment
It does include the blog post's rules on exiting on Ctrl-C, accepting `-` for stdin, disabling color in pipes, and much more.
https://usage.jdx.dev/
Deleted Comment
This is really important. I'd like to expand on this.
Standard output is for the data the program was asked to produce, no more and no less. If user asked for some JSON data, standard output should contain that exact JSON object and absolutely nothing else.
Standard "error" is actually a misnomer. It should have been called the standard user stream. Anything meant for the user to read on the terminal is supposed to go there. Error messages are of course included in that set but so are status messages and verbose output.
This ensures the output of programs can be piped into other programs seamlessly. Non-output data still gets sent to the terminal or redirected somewhere else.
Would have been great if programs were able to easily create new terminal-connected file descriptors for specific purposes. They could document those numbers in their manuals just like they document exit codes. Then users would get "ports" for every output. Could cut down on parsing significantly.
For compatibility, they could all redirect to either standard output or standard error by default... I think I'm gonna experiment with this a bit.
https://learn.microsoft.com/en-us/powershell/module/microsof...
What if programs could define any number of output streams? By default they could all coalesce into the terminal but other programs could connect to each one separately if they needed. Like an audio mixer of sorts: by default you get the mixed audio but there are ways to access each individual voice if needed.
Once you get used to having all that, going back to other shells is pretty hard.
Of course, the real problem is there's no standard, the standards that do exist are ignored, and each new command-line tool generates a new standard.
Can you provide a concrete example of such an application?
That's what it is....a log. Of errors, warnings, informational messages, whatever to complement the primary output.
BONUS: stdlog fits nicely :)
- If this is your first time hearing about the readline/emacs keybindings like Ctrl-E and Ctrl-W, you'll be pleased to know that most macOS input sources use these keybindings. If you're on macOS, feel free to try Ctrl-E, Ctrl-W, or Ctrl-U in your browser's address bar right now
- If you're using a command line program that doesn't support _any_ line editing (e.g. no readline keybindings, and no other keybindings), you can install the `rlwrap` program and launch the REPL under rlwrap. For example Standard ML of New Jersey has a REPL but no line editing functionality, but you can recover that via `rlwrap smlnj`
- "don’t use more than 16 colours" — I would go so far as to say "don't use more than 8 colors, or at least make your colors configurable." Many popular color schemes, including Solaraized and the default Base 16 color scheme, use the "bright" colors to hold various shades of gray. What you think is "bright green" might actually be the same shade of gray that normal text is colored.
Most browsers I've used close the current tab when you press Ctrl-W. Actually, the terminal emulator I use, Alacritty, also does this, and most file explorers that have tabs also do. Iirc even windows explorer does this now, but it's been a while since I've actually used windows.
This can't be right. I use ctrl+w all the time, and occasionally use Alacritty. I'd notice if this shortcut closed my terminal window (it's extremely annoying when I use a web-based ssh, because I have this shortcut deep in my muscle memory).
- Press Ctrl-D, like normal
- Get confused when nothing happens
- Remember that it doesn't work in GHCi, so run `:q` instead
- Get an error message about "lexical error at character '\EOT'", due to Ctrl-D inserting an invisible char at the start of the input
- Try `:q` again, without any invisible prefix
- GHCi successfully quits
Vim works really weird when you start typing random capital letters after thinking you were moving somewhere.
Is there some setting that changes it?
Every other REPL I've used handles this fine, e.g. for quitting Python, Nix repl, SSH sessions, and even the shell itself. Weird.
I mention it because while "yourprogram ~/path/to/file" always works, having a repl that asks for a filename might not work. I've seen a lot of software where this DOES work, so I think it counts as a "most TUI programs do this" thing.
Tilde expansion is the first operation in word expansion:
> The expansions that are performed for a given word shall be performed in the following order: 1) Tilde expansion, parameter expansion, command substitution, and arithmetic expansion shall be performed, beginning to end. 2) Field splitting shall be performed on the portions of the fields generated by step 1. 3) Pathname expansion shall be performed, unless set -f is in effect. 4) Quote removal, if performed, shall always be performed last.
See https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V...
To see the whole shell specification, goto the main page at https://pubs.opengroup.org/onlinepubs/9799919799/. Select "Shell & Utilities" from the top-left frame, then "Shell Command Language" from the bottom-left frame.
In POSIX shells '~name' expands to the home directory of user 'name'
Now your tool need a way to query home directory for any user including system users. Depending on NSS configuration this is more complicated than just reading /etc/passwd.All for the rare case that someone passes a file path starting with tilde and wants it to be expanded. IMO, when you provide a file interactively you do so through a shell and tilde expansion is handled by the shell and otherwise just provide the actual file path and do not rely on the tool doing additional expansions.
PS: I mixed up TUI and CLI program a bit in my head. For an interactively used TUI program it might be beneficial to implement tilde expansion (but then it should be as complete as in Shells). A CLI program should not do magic stuff like tilde expansion.
Going further, as part of shell expansion, expanding the tilde could also be useful for filename expansion in the standard library.
What if there is a file named "~"? What about "~username"? How do you escape/quote it? What if $HOME is not set, or set to something different than the actual user home directory? What about Windows? Also, a lesser known fact is that typing "~." after newline in ssh will force close the connection from the client side and there is nothing the server can do about it, so don't make it part of your workflow.
Entering "~vicky/doc.txt" or "~Ricky/doc.txt" over SSH will actually only write "icky/doc.txt", and entering "~Chris/doc.txt" will have SSH print out it's internal command-line's usage instructions in the middle of the TUI. Screenshot from Vim: https://imgur.com/a/NCt0G9r
Interestingly, bash does do tilde expansion, but not tab completion, after the first equal sign, and after every colon, in variable-style arguments:
While tab completion can be fixed in appropriate contexts using custom completion functions (e.g., [1]), AFAIK there's no way to customize tilde expansion contexts (other than the fact that tilde expanding variable-style arguments is disabled in POSIX mode).[1] https://github.com/scop/bash-completion/blob/8a8880db78e9b04...
https://pubs.opengroup.org/onlinepubs/9799919799/functions/w...
There are conventions, but following all the conventions in a CLI is a lot easier than designing a good GUI. So they tend to be higher quality as a result.
I spend a lot of time thinking how to bring this property to GUIs, but my best answers are still "lots of effort" or "lower your expectations".
The Application Framework defaults which underly most native macOS application build a lot of common keyboard controls in. Use the default menu bar classes with the default basic commands, and Command O, N, Q, X, C,V and probably others come for free, you just implement the code that you need to for those functions. Use a standard text field and you automatically get Command/Option/Fn Left, Right, Up, Down for navigation. It’s more notabke when a macOS application doesn’t follow convention (e.g. InteliJ uses Shift-Opt up/down to move lines rather than expand the selection by paragraph) than when one does.
Windows does decently well on this front, but ctrl as a default modifier can hurt terminal based app usage and there are a number of UI frameworks even within the OS that appear to get different defaults.
And in the Linux world, I think the only way you could do this would be for someone to design (and a distro to standardize on and port apps to) a full on application framework. The window managers don’t want to be in the business of dictating the behavior of stuff in windows. The GUI toolkits don’t want to be in the business of defining os wide defaults and the DEs and distros don’t want to be in the business of if porting or dictating UI frameworks. And realistically there’s no one “on high” that could make the sort of dictation that for example “hence forth copy and paste will be Meta-C and Meta-V”