Loading and ptrace'ing a process on Linux

This is the first post in a series on writing a debugger in C. The project page describes my reasons for staring this project as well as a roadmap for the future posts. Over the course of the series I will develop a complete debugger as a library. But, the goal of this post is to create a minimal debugger that can load and attach an program and run it to completion.

Loading an inferior

The program to be debugged is refered to as the debuggers inferior process or just as “the inferior.” This how I will refer to the inferior process from now on.

Loading an inferior will be done with the underlying execv() call. We will write a function to load and execute the inferior and its signature should be similar to exec() like this:

int execv(const char *path, char *const argv[]);
void dbg_inferior_exec(const char *path, char *const argv[]);

Where path is the path to the file to run as the inferior and argv is the list of command line arguments that will be passed to the inferior.

ptrace

ptrace() is the system call that will allow us to debug the inferior process. It is a single system call with a multitude of different types of requests. ptrace() is declared like this:

long ptrace(enum __ptrace_request request, pid_t pid,
            void *addr, void *data);

Some requests require more information than others so the ptrace() call takes 4 arguments but some argumments may not beused in all requests

For now we only need to look at the PTRACE_TRACEME requeset which can be called in the inferior process to setup the process to be traced. This is not strictly necessary as the debugger could just attach to any running process. But PTRACE_TRACEME causes the inferior to be stopped very early on and allows debugging from the beginning of execution. The PTRACE_TRACEME request ignore all the other arguments passed to ptrace().

You might be wondering how we coerce the inferior into calling ptrace() in the first place. For that I need to explain fork() and exec().

fork and exec

The system calls fork() and exec() (and the exec() variants) are the core to starting processes on POSIX systems. fork() creates a new process but not any process a copy of the processing calling fork(). The analogy is that fork() creates a fork in the road that is process execution.

"fork in the road, loring park" by Connie is licensed under CC BY 2.0

After the fork() the two processes are identical except that

1. They will have different process IDs.
2. The return value of the call to fork() differs in the child and parent.

Using the return value we can decide to do different things in the child process and the parent. Specifcially, from the child process we can make the PTRACE_TRACEME request to setup debugging.

The exec() system call let’s us turn our forked process into any process we want. The proces will continue and will retain it’s process ID but after the exec() call it will load a new binary image into memory and start executing from that image’s entry point (think main function though in reality it is a little more involved than this.)

Fork the inferior

Puting this altogether we have

/* Used for ignored arguments */
static const pid_t ignored_pid;
static const void *ignored_ptr;

static void setup_inferior(const char *path, char *const argv[])
{
  ptrace(PTRACE_TRACEME, ignored_pid, ignored_ptr, ignored_ptr);
  execv(path, argv);
}

static void attach_to_inferior(pid_t pid)
{
  while(1) {

  }
}

void dbg_inferior_exec(const char *path, char *const argv[])
{
  pid_t result;

  do {
    result = fork();
    switch (result) {
    case 0:   // inferior
      setup_inferior(path, argv);
      break;
    case -1:  // error
      break;
    default:  // debugger
      attach_to_inferior(result);
      break;
    }
  } while (result == -1 && errno == EAGAIN);
}

I’ve written 3 functions here. Let’s read over them starting at the bottom and working our way up.

dbg_inferior_exec() is our API. It calls fork() and dispatches to other functions to handle the various cases. The only error handled so far is EAGAIN. We’ll need to add handling for other errors next.

attach_to_inferior() is called from the debugger process to, well, to attach to the inferior. At the moment this function does nothing execpt loop forever. We will enventually fill in this loop and it will become the main loop processing debug events.

setup_inferior() is called after fork() from within the inferior process. First, it makes the PTRACE_TRACEME call to setup the process as the inferior. Note that I have created two global variables used as placeholders for ignored parameters. I pass those through the pid, addr, and data parameters. Then setup_inferior calls execv() to execute the program we wanted as the inferior. After this point the inferior process image will be replaced with the program described by path.

Ready to run!

Just kidding, we have a little more work to do. But, at this point our debugger should be far enough along that we should be able to write a small test and watch and inferior run. But we need those two things first. So we need to

1. Write an inferior to use in our test
2. Write a small test program that uses our API

Our first inferior

Any program should work - even this one:

#include <stdio.h>

int main()
{
  puts("Hello World!");

  return 0;
}

Our first test

Our test program just needs to call our single API, like this:

#include <debugger.h>

int main()
{
  char *argv[1] = { 0 };
  dbg_inferior_exec("./hello", argv);

  return 0;
}

Run it!

Ok, I ran it and … nothing happened. The test is just stuck in the while(1) loop in setup_inferior. “Hello world!” never appears on the screen. What happened?

What happened was that the tracing system stopped our program after the execv(). The ptrace() man page actually describes this if we read the right part. Note, the man page uses the term “tracee” to desribe the process traced by ptrace. This coresponds to what I call the inferior. In the description section of the ptrace man page it says:

If the PTRACE_O_TRACEEXEC option is not in effect, all successful calls
to  execve(2)  by the traced process will cause it to be sent a SIGTRAP
signal, giving the parent a chance to gain control before the new  pro‐
gram begins execution.

and in fact we did not set the PTRACE_O_TRACEEXEC flag so the inferior would have been sent SIGTRAP. Also, earlier the description says:

While  being  traced, the tracee will stop each time a signal is deliv‐
ered, even if the signal is being ignored.  (An exception  is  SIGKILL,
which  has  its usual effect.)  The tracer will be notified at its next
call to waitpid(2) (or one of the related "wait"  system  calls);  that
call  will  return a status value containing information that indicates
the cause of the stop in the tracee.  While the tracee is stopped,  the
tracer  can  use  various  ptrace  requests  to  inspect and modify the
tracee.  The tracer then causes  the  tracee  to  continue,  optionally
ignoring  the  delivered  signal (or even delivering a different signal
instead).

So, at this point the situation we have is that

• The inferior was sent a SIGTRAP signal
• The inferior stoped on the SIGTRAP waiting for the debugger to do something
• The debugger is stuck in while(1) doing nothing.

One thing we can take away from this is that from a TDD perspective we now have a FAILing test. It would be nice if that test were automated and self-checking but we will get to that later. Right now, let’s make our first test PASS.

To correct the situation we have above, our debugger has to take responsibilty for the inferior. It needs to wait for the inferior to receive SIGTRAP and then do something about it.

Waiting for the inferior

We can use the waitpid system call to wait for the SIGTRAP in the inferior. While we are at it, I can check the status of the inferior to confirming my theory that the inferior stopped on a SIGTRAP.

But before I can go off confirming theories, we need to learn how the waitpid system call works. The status it returns contains a lot of information and it can be a little tricky to process.

The function has the following protoype:

pid_t waitpid(pid_t pid, int *status, int options);

We pass the process id that we want to wait for through the pid argument. status is an output parameter and we will ignore options for now and just pass 0.

The interesting quesion is: what does waitpid write to status? status encodes the reason that the inferior stopped in the single int value. To decode the reason a set of macros are provided. The macros we will use in our debugger are:

We can put a few of these macros to use to monitor our inferior in attach_to_inferior like this:

static const void *no_continue_signal = 0;

static void attach_to_inferior(pid_t pid)
{
  while(1) {
    int	status;
    waitpid(pid, &status, 0);

    if (WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP) {
      printf("Inferior stopped on SIGTRAP - continuing...\n");
      ptrace(PTRACE_CONT, pid, ignored_ptr, no_continue_signal);
    } else if (WIFEXITED(status)) {
      printf("Inferior exited - debugger terminating...\n");
      exit(0);
    }
  }
}

Now when I run the test I get this:

$ ./test_exec
Inferior stopped on SIGTRAP - continuing...
Hello World!
Inferior exited - debugger terminating...

I consider this a PASS for our first test.

You will also note that I added a call

ptrace(PTRACE_CONT, pid, ignored_ptr, ignored_ptr);

after determining that the inferior is stopped on SIGTRAP. This call makes the PTRACE_CONT request which requests that the inferior continue execution. The man page describes PTRACE_CONT like this:

Restart  the  stopped tracee process.  If data is nonzero, it is
interpreted as the number of a signal to  be  delivered  to  the
tracee;  otherwise,  no signal is delivered.  Thus, for example,
the tracer can control whether a signal sent to  the  tracee  is
delivered or not.  (addr is ignored.)

Wrapping up

So far we have written a simple program that loads an inferior and debugs it under ptrace(). It detects the start and stop of the inferior process and lets us know what’s happening. While this is a good accomplishment for a single blog post, it isn’t really much of a debugger yet - we can’t inspect any variables when the process is stopped and we can’t set breakpoints to stop at any other locations. But, we have the start of a program that will eventually be able to do all these things. We will continue to explore these topics in future posts.

Also, the program we have isn’t pretty and it’s missing important error checking. It has a test but it isn’t self-checking. I’ll spend the next post fixing these things by getting all the infrastructure for the project setup. This will include build scripts, unit tests, and continuous integeration (the source is already under git source control). After that we can dive into debugger implementation.