File Descriptors

File Descriptors are handles that are used to read and write to files. Of course, since "everything is a file", file descriptors also allow you to do all sorts of things beyond manipulating files.

The file descriptors are just non-negative integers that map to the relevant file via the process's file descriptor table (maintained by the kernel). This table also keeps track of if you're allowed to read or write from the file.

Standard blah

There are three standard file descriptors open at the start of any unix process: standard input, standard output, and standard error, with the corresponding integer values of 0, 1, and 2 respectively.

Reading from standard input allows you to read user input. scanf and gets are really nothing more than wrappers around read(0, ...). Writing from standard output allows your program to output data. printf and puts are really wrappers around write(1, ...). Standard error is used for logging errors.

Open and Close

When you call open with a filepath, it returns a new file descriptor which maps to that new file. File descriptors are always assigned sequentially, so if the first thing you do in a program is open a file, you'll get assigned file descriptor 3.

When you call close with a file descriptor, it removes the mapping from the table. Suppose you plan on running a program as a daemon and you want to printf to a file instead of standard out. If you first close(1, ...) and then open('out.log'...), out.log will get assigned the file descriptor 1, so printf will output to out.log. (As it turns out, this is basically how Unix redirection (echo hi > out) works on the command line).

dup and dup2

dup and dup2 are system calls that create a copy of a file descriptor. dup2 is newer and takes two arguments: the old file descriptor and the desired number of the new file descriptor. (If the new file descriptor is open, it's first closed). dup is older and just takes the old file descriptor as its single argument. It returns the lowest unused descriptor as the duplicate.

CTF challenges are usually run as xinetd services that connect stdin and stdout with the socket so that the challenge can be accessed via netcat. For these pwnables, executing system will launch /bin/sh and allow you to pop a shell.

However, in some pwnables, the binary itself connects to a socket. For these, if you open system, you'll successfully launch a shell, but the shell will live on the server, and you won't be able to send it commands. If however, you first dup2(socket, stdin) and dup2(socket, stdout), /bin/sh will interact with the socket instead of stdin and stdout, and system will do what you want.

What to do if stdin and stdout are closed

Suppose the program closes stdin and stdout before you can carry out your exploit. One thing you can do if you can make system calls is to open a socket to your local machine. This requires a machine with a public ip address and a fair number of ROP gadgets.

What if you can't do that? Well if you are running the process yourself instead of connecting via netcat, you can instead open the special file /dev/tty which connects to the terminal you're in. If instead you are talking via nc, you can connect to the pseudoterminal at /dev/pts/?