My experience with this C build infrastructure is pretty old-school. I’m used to using make and not using unit tests. In my past job we had a monolithic test suite that took around an hour to run. In my current job we have even slower tests. Now, these were integration tests not unit tests and in some cases there are good reasons why they take a long time. But this is going to be a pretty small project (by comparison) and I want to have smaller tests and I want them to run fast.

Choice of tools

cmake

I want more experience with cmake so I will use it for this project. Also, it seems like it simplifes things over make.

No test framework

For now, I’m going to proceed in the system of the test_exec.c that I’ve already written. Simple, standalone C programs that test small pieces of functionality from outside the API. These should still be small and fast but are more end-to-end tests than unit tests. If I find that I need to test internal units I’ll start looking into a way to do that.

I do have to make sure to write self-checking tests.

GitHub

I already have Trap under git but I’ll also store it in github for public availablilty.

Travis CI

I will use Semaphore CIas my cloud CI provider. It has gcc installed and cmake can be installed. It integerates with github. I also considered Travis CI but I’ve used it before and wanted to give Semaphore CI a try.

Writing the Cmakefile

Here’s my initial CMakeList.txt

cmake_minimum_required(VERSION 2.8)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/CMakeModules/")
project(trap)
enable_testing()
add_subdirectory(src)

And this is src/CMakeLists.txt

include_directories(. ../include)
add_library(trap inferior_load.c)

There’s too many steps to running cmake (i.e. three) so I want need a script to run it all for me in one step. I call the script go and at this point it looks like this:

#!/bin/bash

mkdir -p _out
cd _out
cmake ..
make

At this point running ./go builds libtrap.a. So, far so good. But I think I need to make the go script a little more flexible. For example I want to be able to clean, and I’ll probably need to add other commands latter. I also want to keep the script maintainable so my though is to write a function for each target for the script (e.g. “build” or “clean”). Then I will just use the argument to go to call the appropriate function. Here’s the script I have now:

#!/bin/bash

OUTDIR=_out
DEFAULT_TARGET=build
BASEDIR=$(dirname $0)

function main {
    if [ "$1" == "" ]; then
        TARGET=$DEFAULT_TARGET
    else
        TARGET=$1
    fi

    $TARGET
}

function build {
    mkdir -p $OUTDIR
    cd $OUTDIR
    cmake ..
    make
}

function clean {
    rm -rf $OUTDIR
}

cd $BASEDIR
main $*

Now I need to build the tests.

Tests and ctest

First, more cmake for the tests and inferiors. And I need to add the test subdirectory.

enable_testing()
set(LIBS trap)
include_directories(. ../include)
add_executable(test_exec test_exec.c)
target_link_libraries(test_exec ${LIBS})
add_test(test_exec ${CMAKE_CURRENT_BINARY_DIR}/test_exec)

add_subdirectory(inferiors)

This just builds the executable and sets it up as a test. It also adds a subdirectory for the inferiors. Then inferiors/CMakeList.txt is simply:

add_executable(hello hello.c)

With all this, the test passes!

./go
-- Configuring done
-- Generating done
-- Build files have been written to: /home/joseph/src/c/debugger/_out
[ 33%] Built target trap
[ 66%] Built target test_exec
[100%] Built target hello
Test project /home/joseph/src/c/debugger/_out
    Start 1: test_exec
1/1 Test #1: test_exec ........................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.03 sec

But this is wrong! The test shouldn’t be able to find the inferior as the inferior is now in some subdirectory under _out. As I mentioned in the previous post the test is self checking so it doesn’t report any problems. There are three things I need to do here

  1. Fix the test to load the inferior from the right path
  2. Write another test that loads an inferior that doesn’t exist and verifies that some error is returned.
  3. Make test_exec self checking

In another blog I’ve carried this kind of list around in the blog itself. However, since this post is about infrastructure I’ll do something a little more formal. I’ll file github issues for these.

This does mean that I have to push this code to github even though we are really in a failing state. Since we are still geting things setup I guess it’s ok.

GitHub

I created a new repo https://github.com/joekain/trap and pushed to it. Then I created two issues for the TODO items I described above.

GitHub issues

Get the test to pass!

Having this test committed to github with a test that says it passes when it really isn’t working is going to make my head explode! I need to fix it before that happens.

Until I fix #2 and #3 I can look at the test logs to make sure things are working. The most recent test log is stored as _out/Testing/Temporary/LastTest.log. In the run that I have now it says:

Output:
----------------------------------------------------------
Inferior exited - debugger terminating...
<end of output>
Test time =   0.01 sec
----------------------------------------------------------

But, recall from the previous post that we should have this output

Inferior stopped on SIGTRAP - continuing...
Hello World!
Inferior exited - debugger terminating...

If I update the test_exec like this

-  dbg_inferior_exec("./hello", argv);
+  dbg_inferior_exec("./inferiors/hello", argv);

Then it runs correctly and #1 is fixed.

Self-checking tests

test_exec needs a way to verify that the test inferior, hello, actually ran. Test best way for it to do this is to read the stdout from the hello program and verify the text “Hello World!” I do not want to verify they tracing from Trap because it is really just temporary tracing. However, it would be nice to verify that Trap actually stops on SIGTRAP but I will save that for another test at another time because eventually Trap will have callbacks to let the client application know when the inferior stops. This feature is listed on the Roadmap so I won’t add an issue for it now.

We can capture stdout for the inferior by redirecting it to a file. Then we can read back the file and check for the output we want. I’ve written this up like this:

int capturefd(int fd)
{
  char name[] = "test_exec_XXXXXX";
  int captured = mkstemp(name);
  if (dup2(captured, fd) == -1) {
    perror("dup2: ");
  }
  unlink(name);

  return captured;
}

void verify_text(int fd, char *text)
{
  int i;
  char buf[4097] = { 0 };
  lseek(fd, 0, SEEK_SET);
  read(fd, buf, 4096);

  char *p = buf;
  for (i = 0; i < 2 && p < buf + 4096; i++) {
    fprintf(stderr, "Compare aginst: %s\n", p);
    if (strcmp(p, text) == 0) return;
    p += strlen(buf) + 1;
  }

  assert(!"verify_text could not find the specified text");
}

int main()
{
  char *argv[1] = { 0 };

  int captured = capturefd(STDOUT_FILENO);

  dbg_inferior_exec("./inferiors/hello", argv);

  verify_text(captured, "Hello World!\n");

  return 0;
}

I’m expecting at some point to move capturefd and verify_text to some utility file soon. But I’ll wait for some duplication to arrise before doing it so that I can build the right abstractions. Also, I found a bug in the debugger itself. It was extiing when the inferior exited. This prevented me from actually checking the results. I’ve made the following change:

@@ -29,7 +29,7 @@ static void attach_to_inferior(pid_t pid)
       ptrace(PTRACE_CONT, pid, ignored_ptr, no_continue_signal);
     } else if (WIFEXITED(status)) {
       printf("Inferior exited - debugger terminating...\n");
-      exit(0);
+      return;
     }
   }
 }

which allows the test to do some checking and fixes #3.

SemaphoreCI with cmake

The last step for this post is to setup CI with SemaphoreCI. These are the steps I followed to set things up:

  • Sign up
  • Build a new project
  • GitHub
  • Build Public Project
  • Authorize Application
  • Select repository “trap”
  • Select branch “master”
  • Select account “joekain”
  • Read that SemaphoreCI autodetected a C project but doesn’t have full support.
  • Select customize build
  • Add Build Commands
sudo apt-get install cmake
./go

This installs cmake because it doesn’t come standard.

SemaphoreCI says it has “configuration free” but even without the customize build step this was a lot of questions to answer. But, it works! So that’s cool.

The only thing left to do is write a README so I can add a build badge to it. I just need to include this markdown

[![Build Status](https://semaphoreci.com/api/v1/projects/8ea47e1f-3182-4302-8d0d-181e11695861/451420/badge.svg)](https://semaphoreci.com/joekain/trap)

Wrapping Up

Well, the infrastructure is setup now and we have one passing test. From here on out we can start building features.