How the Unix User Interface Works, Part I

In the beginning was the typewriter

The traditional Unix console / terminal / command-line user interface ultimately descends from typewriters. I suppose that the younger set (like your author) mostly haven’t seen typewriters before; they look like this:

source: Wikipedia,

and combine the technologies of moveable type, invented by the Chinese around 1040, with the keyboard, invented by the ancient Greeks in the 3rd century BC, and developed over the course of the 18th and 19th centuries. Moveable “type” is a mass noun; a “piece of type” is an individual hard artifact with a mirror-image of a letter carved into or onto it, capable of transferring ink to paper. The key idea of the a typewriter was to place each piece of type on a lever, and connect the other end to a particular key; pressing the key would move the type forward into a ribbon with ink on it, striking the letter into the paper behind. Other mechanical actions would move the paper (generally) so the next key press would strike the type after the letter just typed. Other keys would be connected to other actions: the Return key would move the paper up and return to the beginning of the line, the Tab key would move the paper to the next (operator-defined) “tab stop”, and so on. A good video of a mechanical typewriter in action can be seen here on YouTube.

The electric typewriter and the teletype

Traditional typewriters were purely mechanical devices: levers and gears were used to translate the mechanical action of pressing the keys into the motion of the type and paper. Once the typewriter became commercially popular, the idea developed of using the keys to send an electrical signal which would drive the type and move the paper. By this point, vertical motion (e.g., moving to the next line) would be supplied by moving the paper up, but the type itself would be placed on a moveable “carriage” and moved right on a normal keypress or left when the user hit Return.

At the same time, the telegraph had been invented, sending coded signals long distances over wires. It was only natural to connect the two: have a keyboard on one end send a signal for the key that was pressed, and interpret the signal on the other end using pieces of type on levers similar to a typewriter’s. The result was the teletype, which is basically a pair of electric typewriters, but with each keyboard connected to the other typewriter’s type bar over a telephone wire. ASCII was originally designed as a code for transmitting characters between teletypes; several of its character codes are designed to control the teletype printer, with the rest designed to format data for transmission.

The printing computer terminal

Once computers were invented, and the idea of using a computer interactively emerged, it was natural to connect a teletype to a computer: pressing a key would send an electrical code to the computer, and the computer could send a matching signal back to the teletype to cause a letter to be printed. This system is partly why the ‘output’ instruction on 1970s languages is called print, and why BASIC had separate PRINT and LPRINT statements, for printing (literally) to the teletype (so to the user) and to a separate line printer, respectively. (Note that a teletype is basically a pair of a keyboard and a line printer anyway.)

There was no echo facility built in to the teletype; the computer had to be programmed explicitly to send each character typed back to the terminal so you could see what you were typing. The simplest way to hide passwords and other input you didn’t want saved forever in the printout was to disable this echo in the computer, which is why that’s what Unix traditionally does.

Enter Unix

The Unix system (sometimes Uɴɪx, never UNIX, except in the trademark) was designed under these conditions. Unix software is designed to process simple text files, with records delimited by the ASCII linefeed or ‘newline’ character: decimal 10, hexadecimal 0x0a, sometimes referred to as ‘control-J’, and designated as \n in C. The kernel (or, rather, the teletype device driver, which is major device number 4 on Linux) was responsible for talking to the teletype, and translating between its conventions and this internal format. It would buffer output as needed, or stop programs that would otherwise overflow the buffer, to slow down output to what the teletype could handle, translate newline characters in program output to whatever the terminal needed to tell it to print a new line (frequently an ASCII carriage return (decimal 13, hexadecimal 0x0d, designated as control-M or \r in C) followed by a newline character), and tra driver, which is major device number 4 on Linux) was responsible for talking to the teletype, and translating between its conventions and this internal format. It would buffer output as needed, or stop programs that would otherwise overflow the buffer, to slow down output to what the teletype could hnslate whatever the terminal keyboard produced when the operator pressed the Enter or Return key (frequently an ASCII carriage return character) into a newline in the program input.

The Unix hater’s handbook complains that Unix ‘should’ have a program that understands the format of its text files and can display them to the user. But it does! Or did, back when it was first designed. That program was the kernel. Having a separate program for displaying each type of file on your computer is the way of madness; you don’t want to go there. The problem is that, once Unix left Bell Labs and entered the University of California at Berkley, that’s exactly where it went.

Video display terminals

This is where the madness truly begins. When video display terminals were first hooked up to a Unix system, the right thing would have been to write a new kernel device driver for them. It could have mapped between the VDT and the Unix tradition of plain text files, possibly even providing better full-screen (WYSIWYG) text editing capabilities, while providing an API for programs that wanted to use the full capabilities of the video terminal to do so and abstracting away from differences between video terminals provided by different companies. The actual capabilities it needed to supply were pretty simple, basically just controlling the position of text output on the screen, due to the limitations of terminals at the time.

Instead, the video display terminals were hooked up to the same teletype (glorified electric typewriter) device driver Unix had been using from the beginning. There is no reason or excuse for this decision.

The kernel kept handling traditional Unix programs, that just wanted to read in plain text files and write out plain text files, in the traditional way. Of course, as terminal technology improved, the kernel’s input editing capabilities stayed the same, which wasn’t good, but things functioned.

Programs that wanted full-screen display capabilities used an ioctl to turn the kernel’s device driver off entirely, so they could talk to the terminal directly. This is where the Unix hater’s handbook starts making more sense, because this was a terrible idea. It’s not portable! Despite the existence of an actual ANSI standard, terminal makers didn’t map the same control sequences to exactly the same behavior, or produce exactly the same byte sequences for the new control keys (like arrow keys).

Programs dealt with this in two ways (simplifying slightly):

  • First, they created a file, called termcap, which could be parsed by full-screen programs to learn (partially) how to program the currently-attached terminal.
  • Then, because it turns out that parsing and using termcap is every full-screen program causes too much code duplication, they wrote a second library, on top of termcap [of course!], called ncurses, for programming terminals.

ncurses has an API that’s probably too large to put into the kernel, but the main point is that its implementation is coupled to the particular terminal it’s talking to, and it depends on bypassing the kernel to talk to it.

Mi computer es su computer

In the meantime, Unix needed to support remote logins, to run programs, including interactive programs, on other computers, at first using rsh, and later of course ssh. Of course, interactive programs expect to talk to a terminal device driver, or possibly to talk to the terminal directly; so how do you make that work over remote connections? BSD chose to do the only (non-) sensible thing: create a new type of device driver, called a pseudo-teletype (pty) to use on the remote computer, put the terminal device driver on the local machine into raw mode (bypassing it), and let the kernel on the remote device handle echoing (and editing) input, translating newlines, etc. If the remote program wanted full-screen capabilities, it put the pty device it was talking to into raw mode.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s