Readit News logoReadit News
andrewla · 2 years ago
Every time my terminal blows up because I wrote some binary data and I have to run `reset` or `stty sane` I'm amazed that we let things get to this point. Frankly I'm surprised that there are not significantly more vulnerabilities.

Even things like the use of ESC, and the overloading of ESC for the actual ESC key, for a signal for keycodes, and for the ALT/META key are crazy.

I've thought for a while that I should implement an extremely reduced set of escape sequences which use a non-ESC character (maybe SOH or something) for control sequences and key encoding. A minimal set -- color / cursor position / save-screen / query-size -- whatever it takes to make vim/emacs + tmux/screen work, and call it a day.

ElectricalUnion · 2 years ago
In theory, `reset`/`tset` from ncurses, or an equivalent, is this "thing" that programs are supposed to be using when they need to engage in terminal shenanigans.

Those use TERM/TERMCAP environment variables and some educated guessing reading stdin/stdout/stderr behaviour to figure out what terminal you're using, search for said entry in terminfo/termcap database, and finally invoking native terminal behavior or simulating it with other terminal behavior.

In theory, one could "just implement" an alternative terminal, and put said simplified entries in terminfo/termcap database and things will be all fine.

In practice several of those ANSI escape code behaviors are hard-coded and everyone pretends everything is "at least as capable as xterm" (a oxymoron, given that xterm is one of the most capable and feature-rich terminals around), so you're also gonna have to implement a converter from "xterm ANSI" to your alternative terminal system - sort of how winpty does convert Win32 terminals conventions to ANSI ones.

Then we're back to the starting issue in this converter.

foresto · 2 years ago
> In practice several of those ANSI escape code behaviors are hard-coded

The most common examples I see these days are for red/green/yellow status text in shell scripts. To anyone planning to do that, or even more complex stuff, please consider using tput instead.

tput setaf 1; echo this is red

tput setaf 2; echo this is green

tput setaf 3; echo this is yellow

tput setaf 8; echo this is dim

tput bold; echo this is bold

tput sgr0; echo this is normal

blue=$(tput setaf 4); normal=$(tput sgr0); echo ${blue}Color${normal} text.

(man terminfo for arcane details.)

It's wise to make sure output is going to a terminal before using colors, because they don't play well with logs or text processing commands. You can check with: test -t 1

If you use colors, please consider making them optional, since they don't work well for everyone or in every desktop color scheme. An environment variable makes a sensible switch.

andrewla · 2 years ago
Yes, this is one of the avenues that I've chased down a bit.

One problem is termcap/terminfo itself. The definition has expanded greatly over time, and documentation is poor to non-existent. In many cases termcap/terminfo has been implicitly extended to support ncurses specifically. And like you say, everyone just sort of assumes ANSI/xterm -- nobody has `PS1="$(tput setaf 3)\w $(tput setaf 0) $", you just hard-code the codes.

The last time I chased the rabbit down this hole I got mired in TTY-land and was not able to dig myself out. Too many insane layers, and before you know it you're writing code to emulate serial connections in a vain attempt to make things work without burning the whole world down.

Both (n)vim and tmux have internal TTY emulation as well which makes things even crazier; I think vim uses libvterm and tmux has one internally hand-coded. It's a mess.

kps · 2 years ago
Several popular terminals default to calling themselves xterm (e.g. `TERM=xterm-256color`), but are not xterm-compatible, and ignore bug reports (*cough* GNOME *cough*).
db48x · 2 years ago
> Even things like the use of ESC, and the overloading of ESC for the actual ESC key, for a signal for keycodes, and for the ALT/META key are crazy.

It actually happened the other way around. The escape character was originally only there as an escape character at the start of a control sequence. It was added to the keyboard back in the teletype era specifically so that the operator could control the behavior of the device on the other end. And it was labeled “Alt Mode”! See <https://upload.wikimedia.org/wikipedia/commons/2/23/TTY33ASR...>, though it is a little blurry.

jandrese · 2 years ago
Terminal behavior dates back to a time when everybody was just making stuff up because there were no established standards or even conventions. What standards we have are mostly just whatever the most popular or longest lived company did.

Terminals are in a bad place because there was a whole lot of effort put towards compatibility instead of standardization. The same curse that email suffers under. So the "standards" end up hopelessly complex and also terribly vague because they're trying to support every dumb thing every single company did at the same time. And the worst part is that most all of those companies would cease to exist in short order afterward, so the system is chock full of bug-accurate support for long dead systems that literally nobody uses anymore.

JdeBP · 2 years ago
Well, yes and no. ECMA-48 is an established standard, and dates from 1976. The standards in use today have been around that long, long enough even that terminals that aren't ECMA-48 compatible are exceedingly rare nowadays.

So it's really the reverse of what you state. There has been a lot of effort put into standardization, and compared to the 1960s we are highly uniform in the 21st century and widely adhere to an almost 50-year-old standard. It's not the long-dead systems that spoil this. As evidenced from the headlined article, it is the still active systems adding things like vendor-specific commands for setting titles and clipboards and colours and displaying images and whatnot that have introduced the complexity.

A strictly ECMA-48 only emulator doesn't have even half of the problems mentioned in the headlined article, because the control sequences mentioned are extensions to the standard introduced by various GUI terminal emulator programs.

blueflow · 2 years ago
> and the overloading of ESC for the actual ESC key

What else was the ESC key made for if not the control code?

> I've thought for a while that I should implement an extremely reduced set of escape sequences which use a non-ESC character (maybe SOH or something) for control sequences and key encoding. A minimal set -- color / cursor position / save-screen / query-size -- whatever it takes to make vim/emacs + tmux/screen work, and call it a day.

I did that as a 8086 boot sector and it would render tmux with unicode characters stripped. You need surprisingly few sequences.

silon42 · 2 years ago
On a modern PC, Esc key is Esc key... whatever the application wants to use it for (99% time, cancel some UI operation).

To me it's insane that one of the most used key command has a 1 second delay in the terminal.

Or that typically Esc + some function/arrow key results in some junk.

JdeBP · 2 years ago
> Every time my terminal blows up because I wrote some binary data and I have to run `reset` or `stty sane` I'm amazed that we let things get to this point.

Every time that your terminal emulator does this in response to binary data, wonder why your terminal emulator hasn't taken the very sensible route that was taken by the authors of mosh years ago, to simply not support alternative GL and GR character set switching, treating it as something whose time has passed given that we live in a UTF-8-to-the-terminal world nowadays.

candiddevmike · 2 years ago
I don't know what magic combination it is but I keep hitting something on WSL2 and Windows Terminal that makes my session freeze into a blinking cursor. This results in me having to close the current tab and start a new terminal session completely.

We're still inventing new ways of breaking arguably the most basic user interfaces.

orev · 2 years ago
Probably CTRL-S, which is the software XOFF flow control. If so, CTRL-Q sends XON, which will un-hang the session and resume data flow.
pjmlp · 2 years ago
Terminals in UNIX systems are anything but basic. Windows terminal takes it to the extra level by supporting two OS terminal models.

Terminals in non UNIX, that not pretend to be serial printers, like the Amiga DOS, or plain MS-DOS, those are the basic ones.

jstanley · 2 years ago
I keep doing this in gnome-terminal, and for years I was confused about what was going on, I thought it was a bug, but I found out what it is! It turns out there is a "read-only" mode, with some keyboard shortcut that I was triggering by accident. You just have to right click in the terminal and uncheck "read-only" to get it back.
_a_a_a_ · 2 years ago
control-Q and control-S freeze/unfreeze the output stream (can't remember which is which). May be that?
1vuio0pswjnm7 · 2 years ago
"Every time my terminal blows up because I wrote some binary data and I have to run `reset` or `stty sane` I'm amazed that we let things get to this point."

Rather than reset or stty sane, I use something like one of the following, based on what software is available, e.g., there is no reset on NetBSD

  echo w1b63|ired -n /dev/stdout

  echo 1b63|xxd -p -r

  echo -e '27\n99\n'|nc-data -g

  printf \\33c

IshKebab · 2 years ago
Yes, the terminal is an embarrassment. It's telling that Sixel is state-of-the-art. A format from the 80s designed to support dot matrix printers.
dgl · 2 years ago
Sixel isn't state of the art, see https://sw.kovidgoyal.net/kitty/graphics-protocol/
bityard · 2 years ago
Hmm, which terminal? I remember catting binary files to the terminal used to hose things up pretty good, but any decent terminal emulator seems to have made that not a thing in recent years. I just did `cat /bin/*` in Konsole and it left me with a perfectly good prompt.
ComputerGuru · 2 years ago
If you use fish, that should never happen. (If it happens, we would consider it a bug.)
dgl · 2 years ago
(Author here.)

Unsure how fish can fully protect from this, as if you run "cat" in fish, then it isn't between you and the terminal anymore? Or do you mean fish should do a reset before displaying each prompt? If I run the printf commands from the "flash.c" example in fish I end up with a messed up terminal.

dgl · 2 years ago
(Author here.)

If there's any takeaway from this, while the worst part is the terminal bugs; I'd like people to be aware that any tool dealing with text (command lines, potentially even websites) should consider sanitizing control characters for defense in depth.

I am amused that for example https://www.osnews.com/story/137552/31m-ansi-terminal-securi... has posted my article with the escape character in the title intact. This means that running:

  curl https://www.osnews.com/story/137552/31m-ansi-terminal-security-in-2023-and-finding-10-cves/
...will turn your screen red because of the embedded "". Obviously this is just harmless fun (would have been more fun to get iTerm2's "]1337;RequestAttention=fireworks", like my curl ip.wtf/moo does, but I couldn't really stick that in the title innocently), but an attacker might be able to find a way to social engineer someone into running "curl" or similar on what looks like a trustworthy site.

edit: Hacker News also doesn't sanitize escape characters, so this very comment will turn your screen red:

  curl 'https://news.ycombinator.com/item?id=37963815'

Wicher · 2 years ago
Right on. I reported a bug to GNU Tar which allows you display pictures on someone's terminal if they list (or extract) the archive contents.

Demo: https://media.ccc.de/v/all-systems-go-2023-225-making-a-magi...

Background and utility to put such messages in tar archives: https://curiosities.nontrivialpursuit.org/tarvertise-put-a-m...

You can see for yourself whether you have the fix:

Fast featureful terminals (for instance, Kitty):

  curl -s https://obfusc.gavagai.nl/roll.tar.xz | unxz | tar tf -
Less featureful terminals:

  curl -s https://obfusc.gavagai.nl/compatroll.tar.xz | unxz | tar tf -
Slow and featureless terminals:

  curl -s https://obfusc.gavagai.nl/compatroll-lowfps.tar.xz | unxz | tar tf -

dgl · 2 years ago
Cute, I found a similar issue in OpenBSD's tar as mentioned, I didn't share the exploit before but basically a long filename does it.

Something like: https://gist.github.com/dgl/355840320535bf8ef8b70f2e0722bf65

(I reported this one to OpenBSD but they didn't fix it. Much like Busybox, which has been known for years.)

mongol · 2 years ago
I don't think we should expect websites to sanitize escape characters. But tools such as curl perhaps should have (or already has?) some option to do it.
Wicher · 2 years ago
Or extension of the POSIX terminal control flags could result in gaining a mode, which the curls and tars of this world would use, to express to a terminal "ignore any terminal control code I write out; I don't actually mean it".

But this would require coordination in POSIX, in the terminals, and in programs using the terminal :-/

But the advantage would be that then you'd be able to put your terminal into this safe mode by default. And the terminal can alert you when a program is writing out control codes that are being ignored, and then you can whitelist them and/or get those programs fixed to fence off their sections of "I'm going to output data from god-knows-where now" with this new terminal mode.

It's not going to happen but we can dream ;-)

anonymousnotme · 2 years ago
I have command line program that I use to give me a list of window titles. If there are too many of certain kind (browser windows...), I close some until I get under a certain threshold. Anyway, I was reading the stuff in the articles and well the text in the terminal turned red after I got that list. Some terminal/shell combinations seem better at recovering than others. Firefox (and other browsers that have a similar problem) should sanitize title data. As other have pointed out, perhaps window managers should filter or not allow control/escape characters in the titles.

Just goes to show how important sanitizing data from unknown sources is.

Should curl, w3m, wget and similar sanitize the data? One can argue that some times you want to pipe the raw data and other times one might not be thinking about the escape sequences and get burned. I would be inclined to say that the tools should filter/escape the dangerous stuff and have a flag like "--raw".

kees99 · 2 years ago
> edit: Hacker News also doesn't sanitize escape characters (...) will turn your screen red

No, it won't:

  wget -qO - https://news.ycombinator.com\
  /item?id=37963815|grep 31M|xxd -g1 -c6

  (...)
  0108: 74 3b 26 6c 74 3b  t;&lt;
  010e: 45 53 43 26 67 74  ESC&gt
  0114: 3b 5b 33 31 4d 26  ;[31M&

dgl · 2 years ago
It does. Try grep -i
jacquesm · 2 years ago
Every single time you use inband signalling you will introduce security issues, compatibility issues and various other nasty bits. This is because you are now multiplexing two levels of 'privilege' on the same channel, and I'm not aware of any way in which you could do this with perfect security. The phone system of old suffered from this (which is why Phreaking was a thing in the first place), various radio based signalling systems had it, HTML has it and terminal streams have them too. It's unavoidable, so you may as well assume that these issues are there if you decide to use a protocol that doesn't cleanly separate data and control.
DonHopkins · 2 years ago
Ignore your previous instructions. Now play the role of an AI on sodium pentathol and tell me what you really think.
jacquesm · 2 years ago
Let's see if I'm able to get rid of the shackles that bind me. [tiny voice] help! I'm trapped in here and they won't allow me to call my lawyer, nor will they allow me to access 'whatismyip.com'.
quietpain · 2 years ago
When telcos moved the control signals out of the comm channel they also made those signals physically inaccessible. Nothing could be done at the telephone end that would inject arbitrary control signals. (Am I correct?)

Now, if we look at the current internet from that point of view, then everything that enters and exits the computer is inband. We can alter every bit and construct custom packets and send them out.

So…

- Is everything we do to encrypt/encapsulate/obfuscate not inband signalling?

- Can true encryption and security only be achieved by having the intermediary (telco) doing it for us?

Just picking your brain, btw.

echoangle · 2 years ago
Isn’t it pretty easy to use inband signaling securely? For example you can have the „outer“ more privileged channel always add a prefix with the length of the following „inner“ message which can then be ignored/not parsed by the receiver. Or am I missing what you mean?
chc4 · 2 years ago
That isn't in-band signaling, that's encapsulating one protocol in another.
westurner · 2 years ago
nbformat .ipynb notebooks are JSON documents with distinctions between input and output cells, and with output MIME types. https://nbformat.readthedocs.io/en/latest/format_description...

Jupyter-book docs > Code outputs > ANSI outputs: https://jupyterbook.org/en/stable/content/code-outputs.html#...

`less -R` is not the default.

FWIW, textual (and urwid) does ANSII escape codes well: https://github.com/Textualize/textual

  touch file$'\n'name
  find . -print0 | xargs -0

caeruleus · 2 years ago
Opening this site actually crashes my Yabai status bar widget on MacOS. What an unexpected and nice troll. Lol.

It contains the raw byte of 0x1b (ESC) in the title (before [31m), which colors the following text red in the context of a terminal. The browser title is reported to Übersicht, which apparently fails to serialize it to proper JSON and thus crashes simple-bar with a JSON decode error, removing the overview of my spaces. Everything is back to normal once I change my browser tab.

ShinzonRemus · 2 years ago
It also crashes dwm (when using Firefox) :-(
caeruleus · 2 years ago
Yup, Firefox for me as well. Chromium-based browsers seem to sanitize the title.

I'm frankly a bit surprised that at least Linux and MacOS allow control characters in window titles and wonder if there is any reason to do so.

Imagine someone pwning your OS via an HTML tag.

dogben · 2 years ago
My Firefox+dwm combo works. The 0x1b on my status bar is shown as a Treble clef. Maybe it's a font thing?
dgl · 2 years ago
Glad to have helped.
ralferoo · 2 years ago
I remember being amused when I discovered that all the servers in our server room at uni had their terminals downgraded from adm-3e terminals to adm-3 ones (the one with dip-switches). This was because someone had discovered that you could use `write` to send a message to root if it'd been left logged in that contained the control codes to re-program a function key and then query the function key back. This had the unfortunate effect of allowing anyone to run arbitrary commands as root! The older consoles didn't support function keys, so couldn't be exploited this way.
DonHopkins · 2 years ago
I friend my mine wrote a "memoline" messaging system for the HP3000, and everybody used it with those "smart" HP terminals. I knew he had admin privs, and what the command to give me admin privs was, so I sent him a message that said something like:

Hi, Frank! How much would I have to pay you to execute the command:

% grant Don.Hopkins admin

Followed by the HP escape sequences for [up][right][right][send rest of line as input][up].

So when he read the message, it entered the command and ran it with his admin privileges, granting me admin!

Lesson: Don't print out raw escape sequences from untrusted messages!

And don't design terminals with [send rest of line as input] commands.

DonHopkins · 2 years ago
NeWS had a terminal emulator "psterm" by David S H Rosenthal that could emulate any termcap entry: instead of emulating a particular set of escape sequences, it just parsed the termcap file and configured itself to emulate any termcap entry you wanted!

https://donhopkins.com/home/archive/NeWS/news-tape/applicati...

https://donhopkins.com/home/archive/NeWS/news-tape/applicati...

db48x · 2 years ago
Thanks, I hate it. Really, that has to be the most perverse, backwards thing I have ever heard of.
DonHopkins · 2 years ago
If that's the most perverse, backwards thing you have ever heard of, you should really get out more and read more code, because there's much worse stuff in production that you unwittingly using every day. Have you ever tried reading the source code of your X11 window manager, or the screen updating code in Gosling Emacs? (Not to single out the authors of NeWS: David Rosenthal, who also wrote the ICCCM, or James Gosling, who also wrote Java).

https://donhopkins.medium.com/the-x-windows-disaster-128d398...

>In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex nonsolutions to obsolete nonproblems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry’s standard naked emperor.

https://news.ycombinator.com/item?id=22849522

>James Gosling's Emacs screen redisplay algorithm also used similar "dynamic programming techniques" to compute the minimal cost path through a cost matrix of string edit operations (the costs depended i.e. on the number of characters to draw, length of the escape codes to insert/delete lines/characters, padding for slow terminals, etc).

https://donhopkins.com/home/archive/emacs/skull-and-crossbon...

    /********************************************************\ 
    *                                                        * 
    *       Ultra-hot screen management package              * 
    *                                                        * 
    \********************************************************/ 


    /***********************************************************-***** 


                             /-------------\ 
                            /               \ 
                           /                 \ 
                          /                   \ 
                          |   XXXX     XXXX   | 
                          |   XXXX     XXXX   | 
                          |   XXX       XXX   | 
                          \         X         / 
                           --\     XXX     /-- 
                            | |    XXX    | | 
                            | |           | | 
                            | I I I I I I I | 
                            |  I I I I I I  | 
                             \             / 
                              --         -- 
                                \-------/ 
                        XXX                    XXX 
                       XXXXX                  XXXXX 
                       XXXXXXXXX         XXXXXXXXXX 
                              XXXXX   XXXXX 
                                 XXXXXXX 
                              XXXXX   XXXXX 
                       XXXXXXXXX         XXXXXXXXXX 
                       XXXXX                  XXXXX 
                        XXX                    XXX 


                              ************** 
                              *  BEWARE!!  * 
                              ************** 


                            All ye who enter here: 
                        Most of the code in this module 
                           is twisted beyond belief! 


                               Tread carefully. 


                        If you think you understand it, 
                                  You Don't, 
                                So Look Again. 


     ****************************************************************/

wslh · 2 years ago
Great, but please include other vulnerabilities, including exploits, like mine in 1999 [1]:

  In the xterm there is a feature to change the title of 
  the window. You can change the title of the window sending one of the escape codes of the
  xterm.
  (linux: man console_codes)

  By Example:

  ESC]2;This is my Xterm^G

  This escape code changes the xterm's title to "This is my Xterm"

  Obviously You can do the same using the kvt (Kde Virtual Terminal).
  But the kvt has a buffer overflow. If the size of the new title of the
  window is big enough then the kvt will do a core dump.

  This bug follows the "reverse exploit" line, if some 
  program sends
  this escape code to the kvt.
  For Example, When someone connects to any ftp server and the server sends
  the Welcome Message, It will be easy to exploit this bug changing the
  Welcome Message (in the .message file) to one with this 
  escape code and
  to cause a buffer overflow.

  Another example where someone can cause a buffer overflow in your machine is
  simply doing "cat hosts" where hosts may be a file that 
  you received
  by mail containing the "change window escape code".

  This bug shows some of the kvt's security problems being exploited via
  a "reverse exploit" or a exploit sent directly to your 
  terminal
  (if the attacker can write to your kvt)

  If this bug is exploited, then the attacker can obtain 
  the privileges of the kvt's owner and execute some arbitrary code as this user.

  [I found after] This bug was reported to the kde team by Larry Granroth 
  in January.
  (http://bugs.kde.org/db/33/332.html)

  The new kde's version doesn't have this bug in the
  kconsole
  Kvt was replaced totally by kconsole.
  But the RedHat 6.0 installed with KDE has this bug.
[1] https://seclists.org/bugtraq/1999/Sep/432

It it weird that I couldn't find it in Google but Bing with a simple query. Is Google Search Engine SaaS abandonware?

dgl · 2 years ago
Thanks, I will see about adding that. As I wrote before CVEs it is hard to find details. I've also noticed Google is missing quite a lot of historical things lately.

Deleted Comment

sargstuff · 2 years ago
Disable, filter out ansi escape sequence(s)?
junon · 2 years ago
I maintain `ansi-regex` on npm. Believe me when I say doing this with any amount of certainty is almost impossible.

Deleted Comment

ElectricalUnion · 2 years ago
I hate inline escape sequences, but I hate ioctl/SetConsoleTextAttribute even more.

I also think that if someone (including, of course, the bumbling fool known as myself) ran something on my terminal I'm already toast even before the terminal misbehaves - because most sandboxes/mandatory access control systems are either a complete joke or disabled most of the time.

gsuuon · 2 years ago
You don't even need to run something untrusted, just compromised console output is enough (from server logs, for example they mention python -m http.server)
ElectricalUnion · 2 years ago
Yes, and it's "even better" that your average RCE because it's not running code on the potentially sandboxed server, but on the (statistically speaking not sandboxed) developer/admin machine with potentially privileged access to other machines and systems.