A System Programming Blog

Demangling the Details: Symbol Resolution in Rusty Trap

Sun, 31 Aug 2025 00:00:00 +0000

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

A few posts ago, we used ownership in Rust to drive the design of our API when setting multiple breakpoints in our rusty_trap debugger. Then we did some vibe coded further refactoring of the API. The API we have for inferiors and breakpoints now is pretty clean and we can move onto the next feature. This has been a long time coming, and I’m proud to announce we are finally going to implement support for looking up symbols from the symbol table when setting breakpoints.

Exploring ELF Files and Symbol Tables

Symbol lookup, or symbol resolution, means that we will no longer have to specify raw addresses for breakpoints. Instead, our users (and tests) can set breakpoints in “twelve::main” or “loop::foo”. rusty_trap will do the work of figuring out what the address is for the breakpoint and then once the address is known the breakpoint can be set and handled the same way we have been doing this entire time.

For reference, I’m starting at commit c8fc68e.

Now, it turns out there are a lot of different symbol tables available. There are dynamic tables used for dynamic linking, static tables, and debug information. For this post we will work with the norm symbol table. Here’s an example of what I mean taken from one of our inferiors. Our inferior (all programs) gets compiled and linked into an object file or some type. The format used on Linux is usually ELF or some variant of ELF. There’s a handy tool called readelf that can show us the content of an ELF object file. readelf -S shows the different sections of the ELF file. Below is the output after running readelf -S on our inferior named twelve and it gives a taste of what we are looking at.

readelf -S twelve
There are 41 section headers, starting at offset 0x37f610:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .interp           PROGBITS         0000000000000350  00000350
       000000000000001c  0000000000000000   A       0     0     1
  [ 2] .note.gnu.pr[...] NOTE             0000000000000370  00000370
       0000000000000020  0000000000000000   A       0     0     8
  [ 3] .note.gnu.bu[...] NOTE             0000000000000390  00000390
       0000000000000024  0000000000000000   A       0     0     4
  [ 4] .note.ABI-tag     NOTE             00000000000003b4  000003b4
       0000000000000020  0000000000000000   A       0     0     4
  [ 5] .gnu.hash         GNU_HASH         00000000000003d8  000003d8
       0000000000000024  0000000000000000   A       6     0     8
  [ 6] .dynsym           DYNSYM           0000000000000400  00000400
       0000000000000660  0000000000000018   A       7     1     8
  [ 7] .dynstr           STRTAB           0000000000000a60  00000a60
       0000000000000412  0000000000000000   A       0     0     1
  [ 8] .gnu.version      VERSYM           0000000000000e72  00000e72
       0000000000000088  0000000000000002   A       6     0     2
  [ 9] .gnu.version_r    VERNEED          0000000000000f00  00000f00
       0000000000000110  0000000000000000   A       7     3     8
  [10] .rela.dyn         RELA             0000000000001010  00001010
       0000000000004740  0000000000000018   A       6     0     8
  [11] .rela.plt         RELA             0000000000005750  00005750
       0000000000000030  0000000000000018  AI       6    28     8
  [12] .init             PROGBITS         0000000000006000  00006000
       000000000000001b  0000000000000000  AX       0     0     4
  [13] .plt              PROGBITS         0000000000006020  00006020
       0000000000000030  0000000000000010  AX       0     0     16
  [14] .plt.got          PROGBITS         0000000000006050  00006050
       0000000000000008  0000000000000008  AX       0     0     8
  [15] .text             PROGBITS         0000000000006060  00006060
       000000000003ceeb  0000000000000000  AX       0     0     16
  [16] .fini             PROGBITS         0000000000042f4c  00042f4c
       000000000000000d  0000000000000000  AX       0     0     4
  [17] .rodata           PROGBITS         0000000000043000  00043000
       0000000000005550  0000000000000000   A       0     0     16
  [18] .debug_gdb_s[...] PROGBITS         0000000000048550  00048550
       0000000000000022  0000000000000001 AMS       0     0     1
  [19] .eh_frame_hdr     PROGBITS         0000000000048574  00048574
       0000000000000fdc  0000000000000000   A       0     0     4
  [20] .eh_frame         PROGBITS         0000000000049550  00049550
       0000000000005700  0000000000000000   A       0     0     8
  [21] .gcc_except_table PROGBITS         000000000004ec50  0004ec50
       0000000000000fac  0000000000000000   A       0     0     4
  [22] .tdata            PROGBITS         0000000000050fe8  0004ffe8
       0000000000000020  0000000000000000 WAT       0     0     8
  [23] .tbss             NOBITS           0000000000051008  00050008
       0000000000000040  0000000000000000 WAT       0     0     8
  [24] .init_array       INIT_ARRAY       0000000000051008  00050008
       0000000000000010  0000000000000008  WA       0     0     8
  [25] .fini_array       FINI_ARRAY       0000000000051018  00050018
       0000000000000008  0000000000000008  WA       0     0     8
  [26] .data.rel.ro      PROGBITS         0000000000051020  00050020
       00000000000025a8  0000000000000000  WA       0     0     8
  [27] .dynamic          DYNAMIC          00000000000535c8  000525c8
       0000000000000210  0000000000000010  WA       7     0     8
  [28] .got              PROGBITS         00000000000537d8  000527d8
       0000000000000828  0000000000000008  WA       0     0     8
  [29] .data             PROGBITS         0000000000054000  00053000
       0000000000000978  0000000000000000  WA       0     0     8
  [30] .bss              NOBITS           0000000000054978  00053978
       00000000000000e8  0000000000000000  WA       0     0     8
  [31] .comment          PROGBITS         0000000000000000  00053978
       0000000000000057  0000000000000001  MS       0     0     1
  [32] .debug_aranges    PROGBITS         0000000000000000  000539cf
       00000000000074c0  0000000000000000           0     0     1
  [33] .debug_info       PROGBITS         0000000000000000  0005ae8f
       00000000000fae6f  0000000000000000           0     0     1
  [34] .debug_abbrev     PROGBITS         0000000000000000  00155cfe
       00000000000010f2  0000000000000000           0     0     1
  [35] .debug_line       PROGBITS         0000000000000000  00156df0
       000000000006830e  0000000000000000           0     0     1
  [36] .debug_str        PROGBITS         0000000000000000  001bf0fe
       0000000000147907  0000000000000001  MS       0     0     1
  [37] .debug_ranges     PROGBITS         0000000000000000  00306a05
       00000000000672c0  0000000000000000           0     0     1
  [38] .symtab           SYMTAB           0000000000000000  0036dcc8
       0000000000004ae8  0000000000000018          39   461     8
  [39] .strtab           STRTAB           0000000000000000  003727b0
       000000000000ccbb  0000000000000000           0     0     1
  [40] .shstrtab         STRTAB           0000000000000000  0037f46b
       000000000000019e  0000000000000000           0     0     1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), l (large), p (processor specific)

There are 40 sections here, the sections are things like .symtab which is the symbol table we will be looking at, .strtab which holds string that can be referenced in other sections (reused strings only get stored once), .data which holds an initialized image of all the global variables. .bss which represents uninitialized (or zeroed) global data but usually doesn’t occupy any space in the object file. The size for .bss given as 0xe8 bytes is likely the size of the section header. There are also .dynsym and .dynstr which are similar to .symtab and .strtab but are used for dynamic linking.

We can take a look at the symbol table using the -s option, though it prints with mangled names. We can use the --demangle=rust option to get nice readable symbols.

$ readelf -s --demangle=rust twelve | head

Symbol table '.dynsym' contains 68 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 (2)
     2: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND [...]@GLIBC_2.2.5 (2)
     3: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND free@GLIBC_2.2.5 (2)
     4: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND _[...]@GLIBC_2.34 (3)
     5: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND abort@GLIBC_2.2.5 (2)
     6: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND _Unw[...]@GCC_3.3 (4)

There are a lot of symbols and it seems that readelf has opted to give us the dynamic symbol table (.dynsym) this may be because our executable is linked PIE and uses, effectively, dynamic loading or at least relocation by default.

We can search for “main” to find:

$ readelf -s --demangle=rust twelve | grep main
   381: 0000000000007820    12 FUNC    LOCAL  DEFAULT   15 twelve::main[...]
   692: 0000000000007830    32 FUNC    GLOBAL DEFAULT   15 main

Now this is all fine and good but we don’t really want to exec readelf and grep from within rusty_trap. Instead, there is a crate for rust called object that can read the object files for us.

Using the object Crate for Symbol Resolution

Taking a look at the object create I see that the read module provides a unified read api with a nice example which lists the sections. It specifically has SymbolTable with a symbols iterator - this might be what we want. Looking deeper, it has symbol_by_name. This is perfect. And of course there is Symbol with name and address Let’s put this into action.

Tests for Symbol Resolution

Instead of writing a new test, instead we’ll update the existing tests to set breakpoints by address. To pass the tests:

Add the object crate to our build.
In trap_inferior_exec following the example from the unified read API and load the inferior file as an Object.
Store the Object in the inferior.
When setting breakpoints

I guess this leaves us with the question of supporting setting breakpoints by address?

If do this it should be a different interface that takes a proper InferiorPointer.
- Oh it actually takes a u64 which is basically InferiorPointer. I guess I couldn’t play so fast and loose with the types in Rust, in C it takes a char * which I could no interpret as a string.
Tests for this would be very fragile, as we’ve seen already. I suppose the test could use object to look up the address and then pass in that address to rusty_trap but that feels a little bit like overkill.
If this is really for educational purposes then I think we don’t need this feature as we have learned this already. But if you want to be able to set breakpoints on raw addresses you should know how to implement it.

Here are the updated tests:

modified   tests/lib.rs
@@ -17,7 +17,7 @@ fn it_can_set_breakpoints() {
 
     let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, 0x000055555555b821);
+    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, "twelve::main");
     let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
@@ -33,7 +33,7 @@ fn it_can_handle_a_breakpoint_more_than_once() {
 
     let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
+    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::foo");
     rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
@@ -50,8 +50,8 @@ fn it_can_handle_more_than_one_breakpoint() {
 
     let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let (inferior, bp_main) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_MAIN);
-    let (inferior, bp_foo) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
+    let (inferior, bp_main) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::main");
+    let (inferior, bp_foo) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::foo");
     let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         if passed_bp == bp_main {

When we run the tests the fail, as expected:

   Compiling rusty_trap v0.1.0 (/home/jkain/projects/rusty_trap)
error[E0308]: mismatched types
  --> tests/lib.rs:20:77
   |
20 |     let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, "twelve::main");
   |                          ----------------------------------------           ^^^^^^^^^^^^^^ expected `u64`, found `&str`
   |                          |
   |                          arguments to this function are incorrect
   |
note: function defined here
  --> /home/jkain/projects/rusty_trap/src/breakpoint/mod.rs:71:8
   |
71 | pub fn trap_inferior_set_breakpoint(
   |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
  --> tests/lib.rs:36:77
   |
36 |     let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::foo");
   |                          ----------------------------------------           ^^^^^^^^^^^ expected `u64`, found `&str`
   |                          |
   |                          arguments to this function are incorrect
   |
note: function defined here
  --> /home/jkain/projects/rusty_trap/src/breakpoint/mod.rs:71:8
   |
71 | pub fn trap_inferior_set_breakpoint(
   |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
  --> tests/lib.rs:53:82
   |
53 |     let (inferior, bp_main) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::main");
   |                               ----------------------------------------           ^^^^^^^^^^^^ expected `u64`, found `&str`
   |                               |
   |                               arguments to this function are incorrect
   |
note: function defined here
  --> /home/jkain/projects/rusty_trap/src/breakpoint/mod.rs:71:8
   |
71 | pub fn trap_inferior_set_breakpoint(
   |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
  --> tests/lib.rs:54:81
   |
54 |     let (inferior, bp_foo) = rusty_trap::trap_inferior_set_breakpoint(inferior, "loop::foo");
   |                              ----------------------------------------           ^^^^^^^^^^^ expected `u64`, found `&str`
   |                              |
   |                              arguments to this function are incorrect
   |
note: function defined here
  --> /home/jkain/projects/rusty_trap/src/breakpoint/mod.rs:71:8
   |
71 | pub fn trap_inferior_set_breakpoint(
   |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0308`.
error: could not compile `rusty_trap` (test "lib") due to 4 previous errors

Passing symbol names through rusty_trap

So, it’s not surprising we need to update the API to take a string with the symbols name. But as we do this let’s rename the current trap_inferior_set_breakpoint because ultimately we will end up setting a breakpoint at an address, that’s just how this works, so we want to keep this functionality. I’ll call it set_breakpoint_at_address.

modified   src/breakpoint/mod.rs
@@ -68,7 +68,7 @@ where
     };
 }
 
-pub fn trap_inferior_set_breakpoint(
+fn set_breakpoint_at_address(
     mut inferior: TrapInferior,
     location: u64,
 ) -> (TrapInferior, TrapBreakpoint) {
@@ -91,3 +91,11 @@ pub fn trap_inferior_set_breakpoint(
 
     (inferior, InferiorPointer(location))
 }
+
+pub fn trap_inferior_set_breakpoint(
+    mut inferior: TrapInferior,
+    location: &str,
+) -> (TrapInferior, TrapBreakpoint) {
+    let address: u64 = 0x55555555b9f4;
+    return set_breakpoint_at_address(inferior, address);
+}

So I have a placeholder address in trap_inferior_set_breakpoint. And of course that causes the test to fail. So, now we need to follow our plan and use the object crate.

Add the object crate to our build.

modified   Cargo.toml
@@ -6,6 +6,7 @@ authors = ["Joseph Kain <joekain@gmail.com>"]
 [dependencies]
 libc = "0.2.174"
 nix = {version = "0.30.1", features = ["process", "ptrace", "signal"]}
+object = "0.37"
 
 [lib]
 name = "rusty_trap"

Our next steps are:

In trap_inferior_exec following the example from the unified read API and load the inferior file as an Object.

Store the Object in the inferior.

This should be, easy, right?

modified   src/inferior/mod.rs
@@ -4,6 +4,9 @@ use libc::pid_t;
 use std::collections::HashMap;
 use std::fmt;
 use std::ops::{Add, Sub};
+use object;
+use std::fs;
+use std::path::Path;
 
 #[derive(Copy, Clone)]
 pub enum InferiorState {
@@ -17,14 +20,17 @@ pub struct Inferior {
     pub pid: pid_t,
     pub state: InferiorState,
     pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
+    obj: object::File,
 }
 
 impl Inferior {
-    pub fn new(pid: pid_t) -> Inferior {
+    pub fn new(pid: pid_t, path: &Path) -> Inferior {
+	let data = fs::read(path).unwrap();
         Inferior {
             pid,
             state: InferiorState::Stopped,
             breakpoints: HashMap::new(),
+	    obj: object::File::parse(&*data).unwrap(),
         }
     }
 }
modified   src/lib.rs
@@ -1,5 +1,6 @@
 extern crate libc;
 extern crate nix;
+extern crate object;
 
 use libc::pid_t;
 use nix::sys::wait::*;
@@ -45,10 +46,10 @@ fn exec_inferior(filename: &Path, _args: &[&str]) {
     unreachable!();
 }
 
-fn attach_inferior(raw_pid: pid_t) -> Result<Inferior, Error> {
+fn attach_inferior(raw_pid: pid_t, filename: &Path) -> Result<Inferior, Error> {
     let nix_pid = Pid::from_raw(raw_pid);
     match waitpid(nix_pid, None) {
-        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior::new(pid.into())),
+        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior::new(pid.into(), filename)),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => Err(e),
     }
@@ -61,7 +62,7 @@ pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<Inferior, Er
                 exec_inferior(filename, args);
                 unreachable!();
             }
-            Ok(ForkResult::Parent { child: pid }) => return attach_inferior(pid.into()),
+            Ok(ForkResult::Parent { child: pid }) => return attach_inferior(pid.into(), filename),
             Err(Error::EAGAIN) => continue,
             Err(e) => return Err(e),
         }

Lifetimes? What is this about lifetimes?

Lifetimes and Self-Referential Structs

error[E0106]: missing lifetime specifier
  --> src/inferior/mod.rs:23:18
   |
23 |     obj: object::File,
   |                  ^^^^ expected named lifetime parameter
   |
help: consider introducing a named lifetime parameter
   |
19 ~ pub struct Inferior<'a> {
20 |     pub pid: pid_t,
21 |     pub state: InferiorState,
22 |     pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
23 ~     obj: object::File<'a>,

Ok, I don’t really understand lifetimes that well. Well, I understand what lifetimes are conceptually: when object A references object B then B must live longer that A, otherwise when B goes out of scope object A would be left with an invalid or dangling reference. And in our particular case I know that I want this object::File to live as long as the inferior. I just don’t have a good understanding of the Rust syntax. That said, I can follow the compiler’s suggestion.

But, I think I can understand that the compiler is saying what I just said I wanted. The lifetime of the Inferior and of the object::File are the same and we express that as lifetime 'a.

Fixing this just unlocks follow on errors:

error[E0726]: implicit elided lifetime not allowed here
  --> src/inferior/mod.rs:26:6
   |
26 | impl Inferior {
   |      ^^^^^^^^ expected lifetime parameter
   |
help: indicate the anonymous lifetime
   |
26 | impl Inferior<'_> {
   |              ++++

error[E0106]: missing lifetime specifier
  --> src/inferior/mod.rs:38:25
   |
38 | pub type TrapInferior = Inferior;
   |                         ^^^^^^^^ expected named lifetime parameter
   |
help: consider introducing a named lifetime parameter
   |
38 | pub type TrapInferior<'a> = Inferior<'a>;
   |                      ++++           ++++

error[E0106]: missing lifetime specifier
  --> src/breakpoint/mod.rs:34:84
   |
34 | fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<&Breakpoint> {
   |                                                               ---------            ^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say which one of `inf`'s 2 lifetimes it is borrowed from
help: consider introducing a named lifetime parameter
   |
34 | fn find_breakpoint_matching_inferior_instruction_pointer<'a>(inf: &'a Inferior<'a>) -> Option<&'a Breakpoint> {
   |                                                         ++++       ++         ++++             ++

error[E0106]: missing lifetime specifier
  --> src/lib.rs:58:69
   |
58 | pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<Inferior, Error> {
   |                                     -----        -------            ^^^^^^^^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `filename` or one of `args`'s 2 lifetimes
help: consider introducing a named lifetime parameter
   |
58 | pub fn trap_inferior_exec<'a>(filename: &'a Path, args: &'a [&'a str]) -> Result<Inferior<'a>, Error> {
   |                          ++++            ++              ++   ++                         ++++

error[E0277]: the trait bound `object::File<'_, &[u8]>: Clone` is not satisfied
  --> src/inferior/mod.rs:23:5
   |
18 | #[derive(Clone)]
   |          ----- in this derive macro expansion
...
23 |     obj: object::File<'a>,
   |     ^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `object::File<'_, &[u8]>`

error[E0515]: cannot return value referencing local variable `data`
  --> src/inferior/mod.rs:29:9
   |
29 | /         Inferior {
30 | |             pid,
31 | |             state: InferiorState::Stopped,
32 | |             breakpoints: HashMap::new(),
33 | |             obj: object::File::parse(&*data).unwrap(),
   | |                                        ---- `data` is borrowed here
34 | |         }
   | |_________^ returns a value referencing data owned by the current function

A couple of these stand out as something more than just propagating the lifetime annotation.

error[E0515]: cannot return value referencing local variable `data`
  --> src/inferior/mod.rs:29:9
   |
29 | /         Inferior {
30 | |             pid,
31 | |             state: InferiorState::Stopped,
32 | |             breakpoints: HashMap::new(),
33 | |             obj: object::File::parse(&*data).unwrap(),
   | |                                        ---- `data` is borrowed here
34 | |         }
   | |_________^ returns a value referencing data owned by the current function

Ok, so the data is read outside of setting up the Inferior struct and then discarded when Inferior::new returns. I guess it is borrowed into object::File::parse? Need to fix that. And the second error that stands out is:

error[E0106]: missing lifetime specifier
  --> src/inferior/mod.rs:38:25
   |
38 | pub type TrapInferior = Inferior;
   |                         ^^^^^^^^ expected named lifetime parameter

I guess on one hand this is just propagation of the lifetime parameter, but on the other this is a leftover wart from some prior refactoring. TrapInferior used to be just a handle that we returned to the user and Inferior was the real struct. Now there is no handle and we return the real struct to the user. And this was a hack to avoid renaming everything. I want to clean this up. Ideally, I would clean this up before making the changes the required addition of lifetimes so that I could compile successfully to verify the rename is correct. I guess I’ll back up and do that first. The way I do this is to:

save the commit so I can cherry-pick it later: f3f9a19
reset back to origin/inferior-ownership-api (starting point for this post)
Verify that build is good
Rename all occurrences
Verify that the build is still good

Ok, this was just a straightforward search and replace.

modified   src/breakpoint/mod.rs
@@ -31,13 +31,13 @@ fn set(inferior: &TrapInferior, bp: &Breakpoint) {
     poke_text(inferior.pid, bp.aligned_address, modified);
 }
 
-fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<&Breakpoint> {
+fn find_breakpoint_matching_inferior_instruction_pointer(inf: &TrapInferior) -> Option<&Breakpoint> {
     let InferiorPointer(ip) = get_instruction_pointer(inf.pid);
     let ip = InferiorPointer(ip - 1);
     return inf.breakpoints.get(&ip);
 }
 
-pub fn handle<F>(inferior: &mut Inferior, callback: &mut F) -> InferiorState
+pub fn handle<F>(inferior: &mut TrapInferior, callback: &mut F) -> InferiorState
 where
     F: FnMut(&TrapInferior, TrapBreakpoint),
 {
modified   src/inferior/mod.rs
@@ -13,15 +13,15 @@ pub enum InferiorState {
 }
 
 #[derive(Clone)]
-pub struct Inferior {
+pub struct TrapInferior {
     pub pid: pid_t,
     pub state: InferiorState,
     pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
 }
 
-impl Inferior {
-    pub fn new(pid: pid_t) -> Inferior {
-        Inferior {
+impl TrapInferior {
+    pub fn new(pid: pid_t) -> TrapInferior {
+        TrapInferior {
             pid,
             state: InferiorState::Stopped,
             breakpoints: HashMap::new(),
@@ -29,8 +29,6 @@ impl Inferior {
     }
 }
 
-pub type TrapInferior = Inferior;
-
 #[derive(Copy, Clone, PartialEq, PartialOrd, Debug, Eq, Hash)]
 pub struct InferiorPointer(pub u64);
 impl InferiorPointer {
modified   src/lib.rs
@@ -45,16 +45,16 @@ fn exec_inferior(filename: &Path, _args: &[&str]) {
     unreachable!();
 }
 
-fn attach_inferior(raw_pid: pid_t) -> Result<Inferior, Error> {
+fn attach_inferior(raw_pid: pid_t) -> Result<TrapInferior, Error> {
     let nix_pid = Pid::from_raw(raw_pid);
     match waitpid(nix_pid, None) {
-        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior::new(pid.into())),
+        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(TrapInferior::new(pid.into())),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => Err(e),
     }
 }
 
-pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<Inferior, Error> {
+pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior, Error> {
     loop {
         match unsafe { fork() } {
             Ok(ForkResult::Child) => {

The build looks good and the tests pass.

Now, I did this in a side branch and I think I can just rebase my current changes on that branch.

$ git checkout object-files
Switched to branch 'object-files'
$ git rebase one-trap-inferior-type
Auto-merging src/inferior/mod.rs
CONFLICT (content): Merge conflict in src/inferior/mod.rs
Auto-merging src/lib.rs
CONFLICT (content): Merge conflict in src/lib.rs
error: could not apply f3f9a19... lifetimes?
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".

Ok, so let’s fix the merge conflicts starting with inferior/mod.rs:

#[derive(Clone)]
pub struct TrapInferior {
    pub pid: pid_t,
    pub state: InferiorState,
    pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
    obj: object::File,
}

<<<<<<< HEAD
impl TrapInferior {
    pub fn new(pid: pid_t) -> TrapInferior {
        TrapInferior {
=======
impl Inferior {
    pub fn new(pid: pid_t, path: &Path) -> Inferior {
	let data = fs::read(path).unwrap();
        Inferior {
>>>>>>> f3f9a19 (lifetimes?)
            pid,
            state: InferiorState::Stopped,
            breakpoints: HashMap::new(),
            obj: object::File::parse(&*data).unwrap(),
        }
    }
}

Ok, so we want the bottom version which we updated with the Path and we need to change Inferior to TrapInferior.

Then the conflict in src/lib.rs:

<<<<<<< HEAD
fn attach_inferior(raw_pid: pid_t) -> Result<TrapInferior, Error> {
    let nix_pid = Pid::from_raw(raw_pid);
    match waitpid(nix_pid, None) {
        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(TrapInferior::new(pid.into())),
=======
fn attach_inferior(raw_pid: pid_t, filename: &Path) -> Result<Inferior, Error> {
    let nix_pid = Pid::from_raw(raw_pid);
    match waitpid(nix_pid, None) {
        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior::new(pid.into(), filename)),
>>>>>>> f3f9a19 (lifetimes?)
        Ok(_) => panic!("Unexpected stop in attach_inferior"),
        Err(e) => Err(e),
    }
}

Again, I want the bottom version that has the filename handling and I’ll have to do the rename.

Ok, git rebase --continue and we are done. Note this does not build right now which is to be expected. We are now back to working on getting the lifetimes right. And it’s pretty much the list of errors we had before minus needing a lifetime on the TrapInferior type alias as we have just removed the alias altogether.

So I want to try to address the one that seems most important first.

error[E0515]: cannot return value referencing local variable `data`
  --> src/inferior/mod.rs:29:9
   |
29 | /         TrapInferior {
30 | |             pid,
31 | |             state: InferiorState::Stopped,
32 | |             breakpoints: HashMap::new(),
33 | |             obj: object::File::parse(&*data).unwrap(),
   | |                                        ---- `data` is borrowed here
34 | |         }
   | |_________^ returns a value referencing data owned by the current function

I ended up spending a lot of of time working on this after writing the above part of the blog. And this is more of a rabbit hole that I expected. At first I wasn’t quite sure what was going on. But it turns out that object::File::parse just builds up references to parts of data and uses that to provide the sections. It’s zero-copy. Cool, that’s efficient. But it means I have to keep the data around for the lifetime of TrapInferior.

My first inclination was to just keep the data in the TrapInferiorlike this:

modified   src/inferior/mod.rs
@@ -21,16 +21,19 @@ pub struct TrapInferior<'a> {
     pub state: InferiorState,
     pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
     obj: object::File<'a>,
+    data: Vec<u8>,
 }
 
 impl TrapInferior {
     pub fn new(pid: pid_t, path: &Path) -> TrapInferior {
 	let data = fs::read(path).unwrap();
+	let obj = object::File::parse(&*data).unwrap();
         TrapInferior {
             pid,
             state: InferiorState::Stopped,
             breakpoints: HashMap::new(),
-            obj: object::File::parse(&*data).unwrap(),
+            obj,
+	    data,
         }
     }
 }

Now, there are a bunch of problems with this. The first, is that I still have data and now also obj on the stack in this function so I can’t return them.

Now, it’s hard to try to get them into the struct together but even if I could it wouldn’t work. I didn’t know this but Rust does not support self-referential structs. That is, one field in a struct can’t contain a reference to another field. I found this both by asking GitHub Copilot and from this Stack Overflow answer. To summarize, the reason Rust does not support self-referential structs is because they can lead to unsafe memory accesses. The recommendation for how to solve this is to have the user call fs::read and pass in the data.

Google Gemini proposed an alternative of using the ouroboros crate which allows creation of self referential structs. However, Gemini recommended that having the user call fs::read is idiomatic Rust.

Coming from languages other than Rust this seemed strange to me. I would favor building an API that takes on the responsibility of reading the file so as to:

Unburden the user from having to do it.
Hide the implementation detail of where the data comes from and what format it is in.

Reading a file isn’t really a detail that needs to be hidden, I suppose. But the fact that it matches the input to object::File::parse is. What if I switched to a different crate one day? Now, it’s also just a vector of bytes so maybe any object parser would want that. But maybe it wouldn’t. In my opinion this couples my API to the implementation a little too tightly.

Gemini points out that the flip side of burdening the user with responsibility is empowering them with control over the lifetime of these objects. Which does sound like a good trade. Gemini provided a number of examples of popular crates that have APIs like this (serde, gimli).

Gemini goes to say

This pattern is a cornerstone of a philosophy in the Rust community called “Parse, Don’t Validate.” Instead of repeatedly passing around raw data (like a &[u8]) and validating it in every function, you parse it once into a structured type (like your TrapInferior<'data>) that guarantees validity through the type system. The lifetime parameter is the mechanism that makes this zero-copy parsing possible and safe.

That makes sense and I buy into “parse, don’t validate”. But, one thing about this bothered me: trap_inferior_exec is where we are currently passed the binary filename. And it will need to be updated to take the binary data as well. So it will look something like:

pub fn trap_inferior_exec(filename: &Path, args: &[&str], data: vec<u8>) -> Result<TrapSession, Error>

This looks great. Well except for one thing, if I’m going to validate up front and then encode all the validated inputs in the type system then do I need to validate that filename and data actually correspond to each other? That is, how do I validate that the user actually called fs::read on the right file? Or should I validate this?

Encapsulating Validated File Names and File Data

If I really wanted to validate it then that would mean calling fs::read inside of trap_inferior_exec and comparing that the data I read matches the data the user passed in. This seems inefficient at best and silly at worse. This is a (proposed) code smell that this is not a good API choice.

When I asked Gemini it agreed that this was overkill (it’s very agreeable) and then ran off to tell me all about how to implement it.

So instead of blindly accepting Gemini’s validation code, I sat back and thought a little bit. Parse, don’t validate–right? What I would really like is for there to be a type, a struct, that encapsulates both the filename and the data. And that type should indicate that the two are in correspondence. I see no reason why such a thing can’t exist. rusty_trap should just provide this type and a API to create it. The type can be created from just the filename and the API can do the fs::read. Then it can return the validated type to the user and the user can in turn pass it (lend) into trap_inferior_exec. The user can control the lifetime of this thing.

Ok, so let’s build that. I guess call it TrapData.

Ideally, I should go back to an earlier commit where tests pass and add this but I’m tired from arguing with Gemini. Let’s just do it.

pub struct TrapData<'a> {
    pub filename: &'a Path,
    pub data: Vec<u8>,
}

impl <'a> TrapData<'a> {
    pub fn new(filename: &Path) -> TrapData {
	TrapData {
	    filename: filename,
	    data: fs::read(filename).unwrap()
	}
    }
}

Now, let’s see if I can implement TrapInferior using this.

Ok, this much of it seems to compile:

#[derive(Clone)]
pub struct TrapInferior<'a> {
    pub pid: pid_t,
    pub state: InferiorState,
    pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
    obj: object::File<'a>,
}

impl <'a> TrapInferior<'a> {
    pub fn new(pid: pid_t, trap_data: &'a TrapData<'a>) -> TrapInferior<'a> {
        TrapInferior {
            pid,
            state: InferiorState::Stopped,
            breakpoints: HashMap::new(),
            obj: object::File::parse(&*trap_data.data).unwrap(),
        }
    }
}

Well, actually it doesn’t because object::File doesn’t implement Clone so TrapInferior can’t derive Clone but I’ll come back to that later.

I think the lifetimes here are correct. For TrapInferior: the object::File lifetime has to be at least as long as the TrapInferior lifetime as a whole.

And for TrapInferior::new we require that TrapData live at least as long as the returned TrapInferior because the TrapInferior will reference the data.

Ok, now we can push this interface change up the call chain.

modified   src/lib.rs
@@ -46,23 +46,23 @@ fn exec_inferior(filename: &Path, _args: &[&str]) {
     unreachable!();
 }
 
-fn attach_inferior(raw_pid: pid_t, filename: &Path) -> Result<TrapInferior, Error> {
+fn attach_inferior<'a>(raw_pid: pid_t, data: &'a TrapData) -> Result<TrapInferior<'a>, Error> {
     let nix_pid = Pid::from_raw(raw_pid);
     match waitpid(nix_pid, None) {
-        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(TrapInferior::new(pid.into(), filename)),
+        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(TrapInferior::new(pid.into(), data)),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => Err(e),
     }
 }
 
-pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior, Error> {
+pub fn trap_inferior_exec<'a>(data: &'a TrapData, args: &[&str]) -> Result<TrapInferior<'a>, Error> {
     loop {
         match unsafe { fork() } {
             Ok(ForkResult::Child) => {
-                exec_inferior(filename, args);
+                exec_inferior(data.filename, args);
                 unreachable!();
             }
-            Ok(ForkResult::Parent { child: pid }) => return attach_inferior(pid.into(), filename),
+            Ok(ForkResult::Parent { child: pid }) => return attach_inferior(pid.into(), data),
             Err(Error::EAGAIN) => continue,
             Err(e) => return Err(e),
         }

Ok, this was pretty straightfoward and seems ok so far. There are some other compile errors to fix and who knows if the next compile pass will find some new problems here after that.

Let’s clean up the leftover errors:

error[E0106]: missing lifetime specifier
  --> src/breakpoint/mod.rs:34:88
   |
34 | fn find_breakpoint_matching_inferior_instruction_pointer(inf: &TrapInferior) -> Option<&Breakpoint> {
   |                                                               -------------            ^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say which one of `inf`'s 2 lifetimes it is borrowed from
help: consider introducing a named lifetime parameter
   |
34 | fn find_breakpoint_matching_inferior_instruction_pointer<'a>(inf: &'a TrapInferior<'a>) -> Option<&'a Breakpoint> {
   |                                                         ++++       ++             ++++             ++

Ok, we need some lifetimes here because TrapInferior just needs one and the returned Breakpoint reference is a borrow from inf.breakpoints.

modified   src/breakpoint/mod.rs
@@ -31,7 +31,7 @@ fn set(inferior: &TrapInferior, bp: &Breakpoint) {
     poke_text(inferior.pid, bp.aligned_address, modified);
 }
 
-fn find_breakpoint_matching_inferior_instruction_pointer(inf: &TrapInferior) -> Option<&Breakpoint> {
+fn find_breakpoint_matching_inferior_instruction_pointer<'a>(inf: &'a TrapInferior) -> Option<&'a Breakpoint> {
     let InferiorPointer(ip) = get_instruction_pointer(inf.pid);
     let ip = InferiorPointer(ip - 1);
     return inf.breakpoints.get(&ip);

error[E0106]: missing lifetime specifier
  --> src/breakpoint/mod.rs:98:7
   |
96 |     mut inferior: TrapInferior,
   |                   ------------
97 |     location: &str,
   |               ----
98 | ) -> (TrapInferior, TrapBreakpoint) {
   |       ^^^^^^^^^^^^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `inferior` or `location`
help: consider introducing a named lifetime parameter
   |
95 ~ pub fn trap_inferior_set_breakpoint<'a>(
96 ~     mut inferior: TrapInferior<'a>,
97 ~     location: &'a str,
98 ~ ) -> (TrapInferior<'a>, TrapBreakpoint) {
   |

This is the nice API where we borrow in and then give back. So it’s borrowed from inferior.

-pub fn trap_inferior_set_breakpoint(
-    mut inferior: TrapInferior,
+pub fn trap_inferior_set_breakpoint<'a>(
+    mut inferior: TrapInferior<'a>,
     location: &str,
-) -> (TrapInferior, TrapBreakpoint) {
+) -> (TrapInferior<'a>, TrapBreakpoint) {
     let address: u64 = 0x55555555b9f4;
     return set_breakpoint_at_address(inferior, address);
 }

Ok, now we are down to this error about clone:

error[E0277]: the trait bound `object::File<'_, &[u8]>: Clone` is not satisfied
  --> src/inferior/mod.rs:38:5
   |
33 | #[derive(Clone)]
   |          ----- in this derive macro expansion
...
38 |     obj: object::File<'a>,
   |     ^^^^^^^^^^^^^^^^^^^^^ the trait `Clone` is not implemented for `object::File<'_, &[u8]>`

Some errors have detailed explanations: E0106, E0277.
For more information about an error, try `rustc --explain E0106`.
error: could not compile `rusty_trap` (lib) due to 3 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `rusty_trap` (lib test) due to 3 previous errors

Ok, so I guess to start with let me ask: why do I need clone?

I think it’s because I have something like this:

pub fn trap_inferior_continue<F>(mut inferior: TrapInferior, mut callback: F) -> (TrapInferior, i32)
where
    F: FnMut(&TrapInferior, TrapBreakpoint)

So that I’m passing these TrapInferior objects around without references and they would shallow copy. As a reminder this is what’s in there:

#[derive(Clone)]
pub struct TrapInferior<'a> {
    pub pid: pid_t,
    pub state: InferiorState,
    pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
    obj: object::File<'a>,
}

Before I added obj we could easily clone this because it’s just some ints and a HashMap that can be cloned. If I got rid of Clone I would have to copy and copying the HashMap seems like a bad idea.

So, what if instead I get rid of Clone and pass by reference? I guess let’s try it.

FWIW just removing clone actually builds (though the tests need some modifications). But, it’s not efficient so let’s keep fixing (honestly maybe that’s not the best choice in real life - get your tests to pass before optimizing).

Interesting, because I’m allowed to pass a reference through a non-reference I can’t use the compiler to find all the places I need to update. But, I think this does it:

modified   src/breakpoint/mod.rs
@@ -68,10 +68,10 @@ where
     };
 }
 
-fn set_breakpoint_at_address(
-    mut inferior: TrapInferior,
-    location: u64,
-) -> (TrapInferior, TrapBreakpoint) {
+fn set_breakpoint_at_address<'a>(
+    mut inferior: &'a mut TrapInferior<'a>,
+    location: u64
+) -> (&'a TrapInferior<'a>, TrapBreakpoint) {
     let aligned_address = location & !0x7u64;
     let target_address = InferiorPointer(location);
     inferior.breakpoints.insert(
@@ -93,9 +93,9 @@ fn set_breakpoint_at_address(
 }
 
 pub fn trap_inferior_set_breakpoint<'a>(
-    mut inferior: TrapInferior<'a>,
+    mut inferior: &'a mut TrapInferior<'a>,
     location: &str,
-) -> (TrapInferior<'a>, TrapBreakpoint) {
+) -> (&'a TrapInferior<'a>, TrapBreakpoint) {
     let address: u64 = 0x55555555b9f4;
     return set_breakpoint_at_address(inferior, address);
 }
modified   src/inferior/mod.rs
@@ -30,7 +30,6 @@ pub enum InferiorState {
     SingleStepping,
 }
 
-#[derive(Clone)]
 pub struct TrapInferior<'a> {
     pub pid: pid_t,
     pub state: InferiorState,
modified   src/lib.rs
@@ -69,7 +69,7 @@ pub fn trap_inferior_exec<'a>(data: &'a TrapData, args: &[&str]) -> Result<TrapI
     }
 }
 
-pub fn trap_inferior_continue<F>(mut inferior: TrapInferior, mut callback: F) -> (TrapInferior, i32)
+pub fn trap_inferior_continue<'a, F>(mut inferior: &'a mut TrapInferior, mut callback: F) -> (&'a TrapInferior<'a>, i32)
 where
     F: FnMut(&TrapInferior, TrapBreakpoint),
 {

Now we just need to fix the tests. I have a bunch of errors like this:

error[E0308]: mismatched types
  --> tests/lib.rs:9:51
   |
9  | ... = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &...
   |       ------------------------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&TrapData<'_>`, found `&Path`
   |       |
   |       arguments to this function are incorrect
   |
   = note: expected reference `&rusty_trap::inferior::TrapData<'_>`
              found reference `&Path`
note: function defined here
  --> /home/jkain/projects/rusty_trap/src/lib.rs:58:8
   |
58 | pub fn trap_inferior_exec<'a>(data: &'a TrapData, args: &[&str]) -> ...
   |        ^^^^^^^^^^^^^^^^^^

There is one of these per test. We need to fix them all by creating a TrapData and passing that instead of the filename.

Ok, that’s done. Now tests compile but fail because they are passing in breakpoint names and we are setting some bogus breakpoint address. E.g. we’ve finally got our types, lifetimes, and references right and can get back to work on adding symbol table support to our debugger. Let’s use the object::File to look up the addresses for the symbol names:

pub fn trap_inferior_set_breakpoint<'a>(
    inferior: &'a mut TrapInferior<'a>,
    location: &str,
) -> (&'a mut TrapInferior<'a>, TrapBreakpoint) {
    let mut address: u64 = 0;
    for symbol in inferior.obj.symbols() {
	let name = symbol.name().unwrap();
	let symbol_address = symbol.address();
	println!("Checking symbol {name} at {symbol_address}");
	if name == location {
	    address = symbol_address;
	    break;
	}
    }

    return set_breakpoint_at_address(inferior, address);
}

Implementing Symbol Resolution

But this never finds the symbol because the symbols look like this:

Checking symbol _ZN4loop3foo17hcca06054783c5707E at 0x7990
Checking symbol _ZN4loop4main17h32a7c34928272921E at 0x79a0

This is the name mangling that I mentioned before. If you are not familiar with these, they are essentially loop::foo and loop::main with additional information encoded such as namespaces, type signatures, and a hash. This ensures uniqueness (think function overloading). It’s also worth noting that the address is the small address and doesn’t include the offset where the binary gets loaded in memory (e.g. the 0x55555… we keep seeing). We’ll need to come back to that. Demangling is the process of converting back to the human readable names and is useful so we can match the names form the symbol table against the function names where we want to set breakpoints.

First, we can use the rustc_demangle crate to decode symbol names. I’m just following the example and ignorign the hash. Ok, this much seems to be working:

Found symbol loop::foo at 0x79c0

Now we need to work on the address. There are two things really:

We need to find the offset to add to get the real address.
It looks like there is something else a little off in that I thought the lower 3 nibbles should match (the page offest) but they don’t. We used to have:

const ADDRESS_OF_FOO: u64 = 0x55555555b9e0;

Let’s start by just taking the offset as a constant and adding it on. That way we can have passing tests and then we can figure out where to get the offset from properly. I’m assuming that both (all) inferiors would have the same offset.

    const BASE: u64 = 0x555555554000;

And the tests pass. Where did I get this address, well I computed

0x55555555b9e0 - 0x79c0

Though this actually works out to 0x555555554020 but that address wouldn’t make any sense. It should aligned to 4KB (4096 or 0x1000 bytes). So I rounded it down. Now, why doesn’t the subtraction give this number to being with? I’m not sure, lihely the inferior has changed since the last time I checked. Let’s check again by loading the inferior in gdb:

Breakpoint 1, 0x000055555555ba20 in main ()
(gdb) b loop::main
Breakpoint 2 at 0x55555555b9d4: file tests/inferiors/loop.rs, line 6.
(gdb) b loop::foo
Breakpoint 3 at 0x55555555b9c0: file tests/inferiors/loop.rs, line 3.

So, it seems they have moved and in fact by 0x20 bytes. This is rather timely as we’ve just implemented a feature (symbol lookup) that our tests immune to these changes.

And now when I do the subtraction it works out properly:

(gdb) p /x 0x55555555b9c0 - 0x79c0
$1 = 0x555555554000

The tests now pass:

running 4 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_set_breakpoints ... ok
test it_can_handle_more_than_one_breakpoint ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Finding the Base Address

Now that we know that having the correct base address will make our tests pass we need to add support to rusty_trap to look it up rather than hardcode it. I poked around at a few different fields in the object file trying to figure out where to get the load address. But ultimately, I ended up asking Bing Copilot:

Me:

When a PIE binary is loaded into memory on Linux to be executed what is the load address? Usually there is address space randomization but I have disabled that using the personality flag before exexcing my process because I am building a debugger. I can find symbol addresses in the ELF object but they need to be offset by the load address. Where should I find this?

Copilot:

When you compile with -pie, the kernel treats your executable as an ET_DYN object and maps its PT_LOAD segments at a runtime-chosen “load base.” With ASLR disabled (ADDR_NO_RANDOMIZE), the kernel uses a fixed, architecture-specific base (on x86_64 it’s traditionally 0x400000).

Why the load base matters Symbol values in the ELF file (st_value) are relative to the link-time base (often 0 for ET_DYN). To turn a symbol’s file offset into the actual virtual address, you add the runtime load base—aka the load bias or l_addr.

Ok, this is basically matches what I understood. Note that there is a lot of jargon here. Let me try to explain:

PIE binary: Position Independent Executable - meaning that the binary can be loaded into any address in memory. Older styles of object files did not do this for the main executable but did for dynamic libraries.
ET_DYN: I think Copilot is saying that the binary gets treated the same way that a dynamic library or other dynamic object would be treated.
PT_LOAD Segment: A segment is a range of an ELF object that gets loaded into memory. It can contain sections. PT_LOAD includes the main binary code sections.

Anyway, I vaguely remember the address 0x400000 being the Linux default load address but that’s clearly not what we are seeing here. Copilot goes on to give some practical advice:

Three ways to discover the load base

/proc/[pid]/maps: Open the target’s /proc/[pid]/maps and locate the entry whose pathname matches your binary. The start address of that VMA is your load base.
Auxiliary vector (auxv): Read /proc/[pid]/auxv and find the AT_PHDR entry (pointer to the in-memory program headers). Compute:
- load_base = AT_PHDR − ELF_HDR.e_phoff**
- where e_phoff comes straight from your ELF header.
Dynamic loader APIs: Use dl_iterate_phdr. In its callback, when dlpi_name[0] == ‘\0’, the field dlpi_addr is the main executable’s base. Alternatively, dladdr(&main, &info) returns info.dli_fbase.

Copilot goes on to give an example of how to implement the first option in C:

uint64_t get_load_base(pid_t pid, const char* exe_path) {
    char filename[64], line[256];
    uint64_t base = 0;
    snprintf(filename, sizeof(filename), "/proc/%d/maps", pid);
    FILE *f = fopen(filename, "r");
    while (fgets(line, sizeof(line), f)) {
        if (strstr(line, exe_path)) {
            sscanf(line, "%lx-", &base);
            break;
        }
    }
    fclose(f);
    return base;
}

This sounds pretty good. And we may need the maps file again later if we want to find any other dynamic libraries that are loaded (if we wanted to set breakpoints in them). Let’s implement a function like this in Rust and then we will call it after we exec and ptrace the inferior and it stops for the first time. That’s our first opportunity to poke around.

I ended up with this function:

// Helper function to look up the base address by inspected /proc/pid/maps
fn get_base_address(pid: pid_t, filename: &Path) -> u64 {
    let proc_filename = format!("/proc/{pid}/maps");
    let file = fs::File::open(proc_filename).expect("Unable to open file");
    let expected = fs::canonicalize(filename).unwrap();
    let expected = expected.to_str().unwrap();
    let reader = BufReader::new(file);
    for line in reader.lines() {
	let line = line.unwrap();
	if line.contains(expected) {
	    let addr_str = line.split('-').next().unwrap();
	    println!("Found base address 0x{addr_str}");
	    return u64::from_str_radix(addr_str, 16).unwrap();
	}
    }
    // This should be an error, there should be error handling.
    println!("Could not find base address for {expected}");
    assert!(false);
    0
}

Which in fact finds the address and prints:

Found base address 0x555555554000

Now we just use this instead of our hard coded version and find that the tests still pass.

Wow, that was a lot of work, but we’ve done it. We’ve implemented symbol look for our inferior and can use it to set breakpoints! Here’s the final set of changes:

modified   src/breakpoint/mod.rs
@@ -100,14 +100,12 @@ pub fn trap_inferior_set_breakpoint<'a>(
 ) -> (&'a mut TrapInferior<'a>, TrapBreakpoint) {
     let mut address: u64 = 0;
 
-    const BASE: u64 = 0x555555554000;
-
     for symbol in inferior.obj.symbols() {
 	let name = format!("{:#}", demangle(symbol.name().unwrap()));
 	let symbol_address = symbol.address();
 	if name == location {
 	    println!("Found symbol {name} at 0x{symbol_address:x}");
-	    address = symbol_address + BASE;
+	    address = symbol_address + inferior.base_address;
 	    break;
 	}
     }
modified   src/inferior/mod.rs
@@ -7,6 +7,7 @@ use std::ops::{Add, Sub};
 use object;
 use std::fs;
 use std::path::Path;
+use std::io::{BufRead, BufReader};
 
 
 pub struct TrapData<'a> {
@@ -35,6 +36,7 @@ pub struct TrapInferior<'a> {
     pub state: InferiorState,
     pub breakpoints: HashMap<InferiorPointer, Breakpoint>,
     pub obj: object::File<'a>,
+    pub base_address: u64,
 }
 
 impl <'a> TrapInferior<'a> {
@@ -44,10 +46,33 @@ impl <'a> TrapInferior<'a> {
             state: InferiorState::Stopped,
             breakpoints: HashMap::new(),
             obj: object::File::parse(&*trap_data.data).unwrap(),
+	    base_address: get_base_address(pid, trap_data.filename),
         }
     }
 }
 
+// Helper function to look up the base address by inspected /proc/pid/maps
+fn get_base_address(pid: pid_t, filename: &Path) -> u64 {
+    let proc_filename = format!("/proc/{pid}/maps");
+    let file = fs::File::open(proc_filename).expect("Unable to open file");
+    let expected = fs::canonicalize(filename).unwrap();
+    let expected = expected.to_str().unwrap();
+    let reader = BufReader::new(file);
+    for line in reader.lines() {
+	let line = line.unwrap();
+	if line.contains(expected) {
+	    let addr_str = line.split('-').next().unwrap();
+	    println!("Found base address 0x{addr_str}");
+	    return u64::from_str_radix(addr_str, 16).unwrap();
+	}
+    }
+    // This should be an error, there should be error handling.
+    println!("Could not find base address for {expected}");
+    assert!(false);
+    0
+}
+
+
 #[derive(Copy, Clone, PartialEq, PartialOrd, Debug, Eq, Hash)]
 pub struct InferiorPointer(pub u64);
 impl InferiorPointer {

I’ve pushed a pull request for Symbol Lookup in breakpoints.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

Lessons Learned

So, we set out implement symbol resolution and along the way learned about lifetimes, self referential structs, and applied systems thinking to our API design (again). In the end we made our debugger much more powerful in and less fragile.

In C, I would have just stuffed everything into one struct, given only a cursory thought to lifetime, and focused on the symbol resolution and breakpoint setting. Rust forces us to think deeply about systems-level problems and that’s a good thing. In C I might think about the lifetimes of the various objects and I might conclude that things look good inside of the debugger but it would be hard to convey those requirements back to the user. Rust makes this explicit and the trade off is that I have to think a little more to make sure it’s going to work both for rusty_tray and for the user of rusty_trap. I have to solve that up front and then I can encode it the type system. This way I solve it on behalf of all the user as well. In C each user must independently think through this proble.. In some way this is like code-usability except here it’s lifetime reusablity. The solution to the lifetime issues for rusty_trap are solved once and reused. The extra thinking required is a good investment.

The self referential structs are also a interesting topic. In C, or maybe C++, I would actually prefer to keep the file data and my uses of the file data together. They are used together so store them together. Their lifetimes are coupled so keep them together. In fact, for my use their lifetimes are basically the same - I only want the file data so I can get an object::File to work with. I suppose though if I wanted to use the file data for something else then coupling tightly together would be a bad idea. I am happy though that we were able to build a nice way to insure that the file name and file data were in correspondence and encode that in the type system.

I like the fact that Rust is pushing my understanding of API design by forcing me to think harder about lifetimes, abstractions, and how information should flow through my systems.

In our next post we’ll see how Rust can inform our design when we work on supporting multithreaded inferiors in our debugger.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

Vibe Coding an API Refactor

Mon, 25 Aug 2025 00:00:00 +0000

In my last post I used ownership to drive the API design for multiple breakpoints, I ended up with an API for the inferiors that I wasn’t entirely happy with. I wrote

Now, I say it’s clean meaning it compiled without any warnings. But I don’t really like the API, it doesn’t feel clean to me. For you rustaceans that have a better knowledge of idiomatic Rust than I do, what’s the right way to create APIs like trap_inferior_exec? I’m dissatisfied with the way it is now because it forces the user to have a mutable Inferior just so that it can pass it back into other rusty_trap functions. The user doesn’t really want to know what’s in there or modify anything. They would depend on rusty_trap to do that for them. So I’d like to return at the very least an immutable object. I guess it’s mutable because they expect rusty_trap to mutate it. Let me know in the comments what you think about this or if you have an suggestions for a cleaner API.

When I had GitHub Copilot with the Claude-3.7 Sonnet Thought model proofread the post it answered this question and suggested:

An opaque handle type that internally manages mutability
A builder pattern for configuration
A more functional approach where operations return new states

I dug in a little deeper and asked

A functional approach is a nice idea but does this imply copying the data in and out of the rusty_trap library? Or do the API calls take ownership and the return the same object back to the caller?

and I think this version of a “functional approach” is a beautiful one, and it aligns well with Rust’s idioms:

let inferior = trap_inferior_exec(...)?;
let inferior = trap_inferior_set_breakpoint(inferior, ADDRESS);
let (inferior, exit_code) = trap_inferior_continue(inferior, callback);

Given that this was Claude’s suggestion I’m going to let it write the code and see how it handles the implementation.

For reference I’m starting at commit e972885.

Me: Initial Ask

I would like to implement the API you suggested

let inferior = trap_inferior_exec(...)?;
let inferior = trap_inferior_set_breakpoint(inferior, ADDRESS);
   let (inferior, exit_code) = trap_inferior_continue(inferior, callback);

Hmm, does trap_inferior_continue need to return the inferior. By the time it has returned the inferior has exited.

Claude: On Returning the Inferior from `trap_inferior_continue`

You raise an excellent question about whether trap_inferior_continue should return the inferior when it has already exited. This depends on your design goals:

Reasons to Not Return the Inferior

The process has terminated, so the primary purpose of the Inferior is complete
It signals more clearly to the user that the debugging session is finished
It’s more memory efficient as Rust will drop the value

fn trap_inferior_continue<F>(inferior: Inferior, callback: F) -> i32
where
    F: FnMut(&Inferior, TrapBreakpoint)
{
    // Implementation that consumes inferior
    exit_code
}

// Usage
let inferior = trap_inferior_exec(path, &[])?;
let inferior = trap_inferior_set_breakpoint(inferior, ADDRESS);
let exit_code = trap_inferior_continue(inferior, callback); // inferior consumed

Reasons to Return the Inferior

Consistency: All API functions follow the same ownership pattern
Post-mortem analysis: The Inferior might contain valuable data even after exit
Future extension: You might add features that work with completed inferiors
Multiple inferiors: In a multi-process debugger, one inferior might exit while others continue

// If you return the inferior:
impl Inferior {
    fn continue_execution<F>(mut self, callback: F) -> (Self, i32)
    where
        F: FnMut(&Self, TrapBreakpoint)
    {
        // Implementation
        self.state = InferiorState::Exited;
        (self, exit_code)
    }
    
    // Then you can add methods that work on exited inferiors
    fn get_memory_dump(&self) -> Option<MemoryDump> {
        if self.state == InferiorState::Exited {
            // Get memory dump from saved state
        } else {
            None
        }
    }
}

Recommended Approach

I’d suggest adding an explicit state for exited inferiors and returning the inferior for consistency. This gives you the most flexibility:

// Add to your InferiorState enum
pub enum InferiorState {
    Running,
    Stopped,
    SingleStepping,
    Exited(i32), // Store the exit code
}

Then users can check the state to determine what operations are valid:

let (inferior, _) = inferior.continue_execution(callback);
match inferior.state {
    InferiorState::Exited(code) => println!("Process exited with code {}", code),
    _ => inferior.continue_execution(callback), // Continue if not exited
}

This approach maintains consistency while making the lifecycle explicit.

An aside: My Thoughts returning the inferior from `trap_inferior_continue`

I think by the time trap_inferior_continue returns the inferior no longer exists so it’s not possible to dump memory or inspect anything. So I’m not going to implement this.

Let’s just move forward with the inferior ownership API.

Me: Let’s move on

Ok, let’s leave the return value as is for now. Can you implement the changes to pass the inferior and ownership into rusty_trap and then return it back to the user via the return value?

Claude: Implementing Ownership-Based Debugger API

Here’s the implementation of the ownership-passing API pattern we discussed. I’ll modify the key functions to take ownership of the inferior and return it back to the user.

Updated `trap_inferior_set_breakpoint` Function

First, let’s modify the breakpoint setting function to take ownership and return a tuple:

pub fn trap_inferior_set_breakpoint(mut inferior: TrapInferior, location: u64) -> (TrapInferior, TrapBreakpoint) {
    let aligned_address = location & !0x7u64;
    let target_address = InferiorPointer(location);
    inferior.breakpoints.insert(
        target_address,
        Breakpoint {
            shift: (location - aligned_address) * 8,
            aligned_address: InferiorPointer(aligned_address),
            target_address,
            original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address)),
        },
    );

    set(&inferior, inferior.breakpoints.get(&target_address).unwrap());

    (inferior, InferiorPointer(location))
}

Updated `trap_inferior_continue` Function

Next, the continue function follows the same pattern:

pub fn trap_inferior_continue<F>(mut inferior: TrapInferior, mut callback: F) -> (TrapInferior, i32)
where
    F: FnMut(&TrapInferior, TrapBreakpoint),
{
    inferior.state = InferiorState::Running;
    ptrace_util::cont(inferior.pid);
    loop {
        inferior.state = match waitpid(Pid::from_raw(inferior.pid), None) {
            Ok(WaitStatus::Exited(_pid, code)) => return (inferior, code),
            Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {
                breakpoint::handle(&mut inferior, &mut callback)
            }
            Ok(WaitStatus::Stopped(_pid, signal)) => {
                panic!(
                    "Unexpected stop on signal {} in trap_inferior_continue.  State: {}",
                    signal, inferior.state as i32
                )
            }
            Ok(_) => panic!("Unexpected stop in trap_inferior_continue"),
            Err(_) => panic!("Unhandled error in trap_inferior_continue"),
        };
    }
}

Updating the `breakpoint::handle` Function

We need to modify this function to borrow the inferior instead of taking ownership:

pub fn handle<F>(inferior: &mut Inferior, callback: &mut F) -> InferiorState
where
    F: FnMut(&TrapInferior, TrapBreakpoint),
{
    let bp = find_breakpoint_matching_inferior_instruction_pointer(inferior)
        .expect("Could not find breakpoint");

    match inferior.state {
        InferiorState::Running => (),
        _ => panic!("Unhandled error in breakpoint::handle"),
    }
    callback(inferior, bp.target_address);
    step_over(inferior, bp);
    return match waitpid(Pid::from_raw(inferior.pid), None) {
        Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {
            set(inferior, bp);
            cont(inferior.pid);
            InferiorState::Running
        }
        Ok(WaitStatus::Exited(_pid, _code)) => InferiorState::Running,
        Ok(WaitStatus::Stopped(_pid, signal)) => {
            panic!(
                "Unexpected stop on signal {} in breakpoint::handle.  State: {}",
                signal, inferior.state as i32
            )
        }
        Ok(_) => panic!("Unexpected stop in breakpoint::handle"),
        Err(_) => panic!("Unhandled error in breakpoint::handle"),
    };
}

Updated Test Example

Here’s how the tests would change to use the new API:

#[test]
fn it_can_set_breakpoints() {
    let mut breakpoint_count: i32 = 0;

    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
    let expected_pid = inferior.pid;
    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, 0x000055555555b821);
    let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
        assert_eq!(passed_inferior.pid, expected_pid);
        assert_eq!(passed_bp, bp);
        breakpoint_count += 1;
    });

    assert_eq!(breakpoint_count, 1);
}

Multiple Breakpoints Test

Here’s the updated multiple breakpoints test:

#[test]
fn it_can_handle_more_than_one_breakpoint() {
    let mut bp_main_count: i32 = 0;
    let mut bp_foo_count: i32 = 0;

    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
    let expected_pid = inferior.pid;
    let (inferior, bp_main) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_MAIN);
    let (inferior, bp_foo) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
    let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
        assert_eq!(passed_inferior.pid, expected_pid);
        if passed_bp == bp_main {
            bp_main_count += 1;
        } else if passed_bp == bp_foo {
            bp_foo_count += 1;
        } else {
            panic!(
                "Unexpected breakpoint {} encountered.  Expected {} or {}",
                passed_bp, bp_main, bp_foo
            );
        }
    });

    assert_eq!(bp_main_count, 1);
    assert_eq!(bp_foo_count, 5);
}

This API pattern provides several benefits:

Makes ownership transfers explicit
Eliminates mutable references to shared state
Creates a more fluent and natural API
Enables method chaining in the future (with method implementations)
Makes it harder to use the API incorrectly

The main downside is that we need to reassign the inferior at each step, but this clearly communicates the ownership transfer semantics, which is a good thing in Rust.

Aside: Review

That looks pretty good. I think I had mentioned to Claude earlier (before this post) that I don’t think trap_inferor_set_breakpoint should return the breakpoint anymore. This was in the example I gave Claude up front. The reason this is unnecessary is that the user now just uses the address as the handle of the breakpoint. But maybe that would be weird actually, trap_inferor_set_breakpoint shouldn’t only return and inferior as that would be confusing. I’d think it just returns a breakpoint as that’s what it did before.

There were two other tests that Claude left for me as an exercises. I could ask it to fix them but it doesn’t seem worth the time. But that’s one thing I find is that often the AI leaves me a lot of the work. I thought the point was for it to do work for me. It often seems more like it’s some Senior engineer helping me out and leaving me to do the grunt work…

I plugged everything in and ran the tests and they pass:

    Running tests/lib.rs (/home/jkain/projects/rusty_trap/target/debug/deps/lib-a0fc739a67a23e62)

running 4 tests
test it_can_exec ... ok
test it_can_set_breakpoints ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_handle_more_than_one_breakpoint ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

And that’s pretty nice. Claude gave us a good suggestion and was able to implement it for us. Here’s it’s changes as a diff for reference:

modified   src/breakpoint/mod.rs
@@ -68,7 +68,7 @@ where
     };
 }
 
-pub fn trap_inferior_set_breakpoint(inferior: &mut TrapInferior, location: u64) -> TrapBreakpoint {
+pub fn trap_inferior_set_breakpoint(mut inferior: TrapInferior, location: u64) -> (TrapInferior, TrapBreakpoint) {
     let aligned_address = location & !0x7u64;
     let target_address = InferiorPointer(location);
     inferior.breakpoints.insert(
@@ -81,7 +81,7 @@ pub fn trap_inferior_set_breakpoint(inferior: &mut TrapInferior, location: u64)
         },
     );
 
-    set(inferior, inferior.breakpoints.get(&target_address).unwrap());
+    set(&inferior, inferior.breakpoints.get(&target_address).unwrap());
 
-    InferiorPointer(location)
+    (inferior, InferiorPointer(location))
 }
modified   src/lib.rs
@@ -68,7 +68,7 @@ pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<Inferior, Er
     }
 }
 
-pub fn trap_inferior_continue<F>(inferior: &mut TrapInferior, callback: &mut F) -> i32
+pub fn trap_inferior_continue<F>(mut inferior: TrapInferior, mut callback: F) -> (TrapInferior, i32)
 where
     F: FnMut(&TrapInferior, TrapBreakpoint),
 {
@@ -76,9 +76,9 @@ where
     ptrace_util::cont(inferior.pid);
     loop {
         inferior.state = match waitpid(Pid::from_raw(inferior.pid), None) {
-            Ok(WaitStatus::Exited(_pid, code)) => return code,
+            Ok(WaitStatus::Exited(_pid, code)) => return (inferior, code),
             Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {
-                breakpoint::handle(inferior, callback)
+                breakpoint::handle(&mut inferior, &mut callback)
             }
             Ok(WaitStatus::Stopped(_pid, signal)) => {
                 panic!(
modified   tests/lib.rs
@@ -6,23 +6,19 @@ const ADDRESS_OF_FOO: u64 = 0x55555555b9e0;
 
 #[test]
 fn it_can_exec() {
-    let mut inferior =
-        rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
-    assert_eq!(
-        12,
-        rusty_trap::trap_inferior_continue(&mut inferior, &mut |_, _| {})
-    );
+    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
+    let (_inferior, exit_code) = rusty_trap::trap_inferior_continue(inferior, |_, _| {});
+    assert_eq!(12, exit_code);
 }
 
 #[test]
 fn it_can_set_breakpoints() {
     let mut breakpoint_count: i32 = 0;
 
-    let mut inferior =
-        rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
+    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, 0x000055555555b821);
-    rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
+    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, 0x000055555555b821);
+    let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
         breakpoint_count += 1;
@@ -35,11 +31,10 @@ fn it_can_set_breakpoints() {
 fn it_can_handle_a_breakpoint_more_than_once() {
     let mut breakpoint_count: i32 = 0;
 
-    let mut inferior =
-        rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
+    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_FOO);
-    rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
+    let (inferior, bp) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
+    rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
         breakpoint_count += 1;
@@ -53,12 +48,11 @@ fn it_can_handle_more_than_one_breakpoint() {
     let mut bp_main_count: i32 = 0;
     let mut bp_foo_count: i32 = 0;
 
-    let mut inferior =
-        rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
+    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp_main = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_MAIN);
-    let bp_foo = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_FOO);
-    rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
+    let (inferior, bp_main) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_MAIN);
+    let (inferior, bp_foo) = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
+    let (_, _) = rusty_trap::trap_inferior_continue(inferior, |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         if passed_bp == bp_main {
             bp_main_count += 1;

Takeaways

I guess I’m taking two things away from this experience:

I should remember this Rust idiom of taking ownership of a passed argument and returning it give the object back to the caller. That’s a clean way of managing the ownership.
I can just vibe code this whole thing.

I joke about vibe coding the entire project but I have been using AI tools in my job a lot lately and I find they are pretty powerful. They may need to be pointed in the right direction and their work should be reviewed but they often surprise me both in terms of the code they write and also in terms of the insight they can provide. I plan to write more about AI tools in the future.

Just to close this out I’ve pushed out a PR:

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

Multiple Breakpoints in Rust: Ownership-Driven Debugger Design

Sun, 17 Aug 2025 00:00:00 +0000

Welcome back to my series on writing a debugger in rust. In our previous post we updated our custom breakpoint handling to support multiple breakpoints in C. And in this post we will do the same thing in Rust.

For reference I’m starting at commit a9447ec.

But this is Rust and not C and we will follow Rust’s best practices which will guide us toward an implementation that eliminates global state and models ownership properly and makes the relationship between inferiors and their breakpoints explicit in the type system. What started off as a C-style port became an ownership-driven rearchitecture.

I started this journey with a new test:

#[test]
fn it_can_handle_more_than_one_breakpoint() {
    let mut bp_main_count: i32 = 0;
    let mut bp_foo_count: i32 = 0;

    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
    let bp_main = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_MAIN);
    let bp_foo = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
    rusty_trap::trap_inferior_continue(inferior, &mut |passed_inferior, passed_bp| {
        assert_eq!(passed_inferior, inferior);
        if passed_bp == bp_main {
            bp_main_count += 1;
        } else if passed_bp == bp_foo {
            bp_foo_count += 1;
        }
    });

    assert_eq!(bp_main_count, 1);
    assert_eq!(bp_foo_count, 5);
}

This test is similar to our other tests except that those tests set either a breakpoint in main or in foo and this test does both. Then in the lambda / callback it checks which breakpoint was reached and increments the corresponding counter. When it’s all done it checks for expected counts. main is called once and triggers it’s breakpoint once. foo is called in a loop so it triggers 5 times.

So, what do I need to do to make it pass?

We need to allocate breakpoints instead of using one global.
- Put off dynamic allocation until I work through deleting as well.
We need that fancy version of breakpoint_handle that we restructured in the C version.

When writing breakpoint::handle I wanted to mimic what is in the C version and assert(state == INFERIOR_RUNNING);

error[E0369]: binary operation `==` cannot be applied to type `inferior::InferiorState`
  --> src/breakpoint/mod.rs:52:23
   |
52 |     assert!(inf.state == InferiorState::Running);
   |             --------- ^^ ---------------------- inferior::InferiorState
   |             |
   |             inferior::InferiorState
   |
note: an implementation of `PartialEq` might be missing for `inferior::InferiorState`
  --> src/inferior/mod.rs:6:1
   |
6  | pub enum InferiorState {
   | ^^^^^^^^^^^^^^^^^^^^^^ must implement `PartialEq`
help: consider annotating `inferior::InferiorState` with `#[derive(PartialEq)]`
  --> src/inferior/mod.rs:6:1
   |
6  + #[derive(PartialEq)]
7  | pub enum InferiorState {
   |

So instead I wrote:

    match inf.state {
	InferiorState::Running => (),
	_ => panic!("Unhandled error in breakpoint::handle"),
    }

Which works, and I guess I realized this would work because the old version matched on inf.state to check for Running vs SingleStepping. But, is this really what I have to do? And what does match do that’s different than ==?

Anyway, here’s the completed breakpoint::handle

pub fn handle<F>(inf: Inferior, callback: &mut F) -> InferiorState
where
    F: FnMut(TrapInferior, TrapBreakpoint),
{
    let inferior = inf.pid;
    let (user_breakpoint, bp) =
        find_breakpoint_matching_inferior_instruction_pointer(inf)
        .expect("Could not find breakpoint");

    match inf.state {
        InferiorState::Running => (),
        _ => panic!("Unhandled error in breakpoint::handle"),
    }
    callback(inferior, user_breakpoint);
    step_over(inferior, bp);
    return match waitpid(Pid::from_raw(inf.pid), None) {
        Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {
            set(inferior, bp);
            cont(inferior);
            InferiorState::Running
        }
        Ok(WaitStatus::Exited(_pid, _code)) => InferiorState::Running,
        Ok(WaitStatus::Stopped(_pid, signal)) => {
            panic!(
                "Unexpected stop on signal {} in breakpoint::handle.  State: {}",
                signal, inf.state as i32
            )
        }
        Ok(_) => panic!("Unexpected stop in breakpoint::handle"),
        Err(_) => panic!("Unhandled error in breakpoint::handle"),
    };
}

Recall, the goal here is to both handle the initial breakpoint and single step in one call to breakpoint::handle. This seems to work for the new test. And it seems to work for each test individually but it doesn’t work when I run all the tests:

running 4 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_handle_more_than_one_breakpoint ... FAILED
test it_can_set_breakpoints ... FAILED

failures:

---- it_can_handle_more_than_one_breakpoint stdout ----

thread 'it_can_handle_more_than_one_breakpoint' panicked at tests/lib.rs:64:5:
assertion `left == right` failed
  left: 0
 right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- it_can_set_breakpoints stdout ----

thread 'it_can_set_breakpoints' panicked at tests/lib.rs:28:5:
assertion `left == right` failed
  left: 0
 right: 1


failures:
	it_can_handle_more_than_one_breakpoint
	it_can_set_breakpoints

test result: FAILED. 2 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

This feels again like the tests are being run concurrently but I’m still running cargo test -- --test-threads 1. Can I confirm test are running sequentially by putting a mutex in the tests?

I mutexed the tests based on this post.

modified   tests/lib.rs
@@ -1,65 +1,89 @@
 extern crate rusty_trap;
 use std::path::Path;
+use std::sync::Mutex;
 
 const ADDRESS_OF_MAIN: u64 = 0x55555555b9f4;
 const ADDRESS_OF_FOO: u64 = 0x55555555b9e0;
 
+static LIBRARY_USER: Mutex<Executor> = Mutex::new(Executor);
+
+#[derive(Clone, Copy)]
+struct Executor;
+
+impl Executor {
+    fn run_test(self, f: impl FnOnce()) {
+        f();
+    }
+}
+
 #[test]
 fn it_can_exec() {
-    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
-    assert_eq!(
-        12,
-        rusty_trap::trap_inferior_continue(inferior, &mut |_, _| {})
-    );
+    LIBRARY_USER.lock().unwrap().run_test(|| {
+        let inferior =
+            rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
+        assert_eq!(
+            12,
+            rusty_trap::trap_inferior_continue(inferior, &mut |_, _| {})
+        );
+    });
 }

I’ve only shown the first test, the rest are wrapped in this executor the same way. The tests still fail which I guess is expected. But the mutexing also fails? Here are the test failures:

---- it_can_handle_more_than_one_breakpoint stdout ----

thread 'it_can_handle_more_than_one_breakpoint' panicked at tests/lib.rs:87:9:
assertion `left == right` failed
  left: 0
 right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- it_can_set_breakpoints stdout ----

thread 'it_can_set_breakpoints' panicked at tests/lib.rs:33:25:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }

Oh, ok this makes sense the rust docs for Mutex

The mutexes in this module implement a strategy called “poisoning” where a mutex is considered poisoned whenever a thread panics while holding the mutex. Once a mutex is poisoned, all other threads are unable to access the data by default as it is likely tainted (some invariant is not being upheld).

So when the first test failed it panicked and poisoned the mutex. Then the next test that tries to acquire the mutex finds it poisoned and panics as well. I think this does confirm the tests are running synchronously.

So then what’s wrong?

Is there some state being carried, with the rusty_trap library, across tests?
- Of course there is some state, we have a global vector of breakpoints after all. But it should work well enough to run 3 tests back to back, right?
- But I don’t want to rule it out. I mean I think it should work but I also the whole system would work and it doesn’t so something’s wrong in my understanding.

When things don't work as we expect it means we misunderstand something about the system. When this happens question everything, trust nothing you think you know.

That said, I hope the tests are correct. I removed the mutexing since it doesn’t make a difference - the tests are simpler without it. And, I’d rather see the test fail than the mutex fail. Next, I want to use these tests to learn more about the problem. The errors look like this:

running 4 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_handle_more_than_one_breakpoint ... FAILED
test it_can_set_breakpoints ... ok

failures:

---- it_can_handle_more_than_one_breakpoint stdout ----

thread 'it_can_handle_more_than_one_breakpoint' panicked at tests/lib.rs:64:5:
assertion `left == right` failed
  left: 0
 right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
	it_can_handle_more_than_one_breakpoint

test result: FAILED. 3 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

This happens the same way every time which is expected. And again, if I were to comment out the other tests then it_can_handle_more_than_one_breakpoint would pass.

What the failure says is that the breakpoint in main was hit once as expected but the breakpoint in foo was never hit. One thing that the test doesn’t capture is if we stopped at the wrong address. The test should check for this, if this did happen that should also count as a test failure and it should be called out differently.

Here’s a test that does that:

#[test]
fn it_can_handle_more_than_one_breakpoint() {
    let mut bp_main_count: i32 = 0;
    let mut bp_foo_count: i32 = 0;

    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
    let bp_main = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_MAIN);
    let bp_foo = rusty_trap::trap_inferior_set_breakpoint(inferior, ADDRESS_OF_FOO);
    rusty_trap::trap_inferior_continue(inferior, &mut |passed_inferior, passed_bp| {
        assert_eq!(passed_inferior, inferior);
        if passed_bp == bp_main {
            bp_main_count += 1;
        } else if passed_bp == bp_foo {
            bp_foo_count += 1;
        } else {
            panic!(    // How do I write this as an assertion?  Or should I?
                "Unexpected breakpoint {} encountered.  Expected {} or {}",
                passed_bp, bp_main, bp_foo
            );
        }
    });

    assert_eq!(bp_main_count, 1);
    assert_eq!(bp_foo_count, 5);
}

And when I run this test I get this (along with the failure):

Unexpected breakpoint 0 encountered.  Expected 1 or 2

Recall that the breakpoints passed out to the user are just indices into GLOBAL_BREAKPOINTS. So, something is getting crossed where one of the breakpoints from the previous test is being hit.

Another issue is that we have global_inferior but the 4 different tests call exec_inferior 4 different times. So this is likely one problem though I would think that global_inferior would be overwritten and it would be fine (I’m mean, it’s an awful code smell but I don’t think it would lead to this bug)

Another thought is that somewhere I forgot to update breakpoint handling to pick the right breakpoint and so we find the wrong one.

Oh, as I write that down I realize that find_breakpoint_matching_inferior_instruction_pointer is wrong. Remember, it looks like this:

fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<(TrapBreakpoint, Breakpoint)> {
    let ip = ptrace_util::get_instruction_pointer(inf.pid).as_i64();
    unsafe { for (idx, bp) in GLOBAL_BREAKPOINTS.iter().enumerate() {
	if bp.target_address.as_i64() == ip - 1 {
	    return Some((idx, *bp))
	}
    } };
    None
}

So, it does this linear search through GLOBAL_BREAKPOINTS until it finds a matching target_address. Well, most of the breakpoints in here have the same target address because we are running unit tests that all break in loop:foo which has the same address every time.

This is a mess and just shows one of the main problems with global variables that Rust is trying to solve. And I worked around all of Rust’s efforts by putting unsafe everywhere and paid the price.

So let’s do this right and get rid of GLOBAL_BREAKPOINTS. Earlier I said that I would

Put off dynamic allocation until I work through deleting as well.

So let’s think about deleting. I had been thinking to mimic the C API in which there is a trap_inferior_remove_breakpoint function that the user can call to delete their breakpoints. But this is a very C style API. Between trap_inferior_set_breakpoint and trap_inferior_remove_breakpoint it basically says: “Hi user, here’s your breakpoint it’s your problem now. Free it, lose it, whatever. I don’t care”. That’s not a Rusty attitude.

In Rust I guess the right idea here is to give the user a breakpoint that they own and that can clean itself up when it goes out of scope. Also it should not be a reference to anything global or unsafe. There is a second piece to this which is that we can’t delete the breakpoint while the inferior is running. So the C version sets the breakpoint in this deferred removal state and cleans it up later. I don’t know if I want to do this in Rust the same way. One thing I don’t like about setting this deferred stat is that it is the only mutation the breakpoint goes through. The breakpoints could be immutable if we do this a different way. This also feels like reference counting: the user has one reference and a running inferior with the breakpoint installed has another reference. Rust has reference counting support so I’ll keep that in mind.

Anyway, I don’t really want to get into the feature of deleting breakpoints right now because we currently have a failing test and should fix that first. Deleting breakpoints will require new tests. So let’s at least start with a basic dynamically allocated breakpoint and worry about proper deletion afterwards. I think none of the tests we have at the moment will trigger a problem with deleting a breakpoint while the inferior is running.

So I think I want to Box up a new breakpoint?

First we need to get rid of Breakpoint and TrapBreakpoint or TrapBreakpoint is Box?

I think something like this would work:

struct Breakpoint {
    shift: u64,
    target_address: InferiorPointer,
    aligned_address: InferiorPointer,
    original_breakpoint_word: i64,
}

pub type TrapBreakpoint = Box<Breakpoint>;

// ...

pub fn trap_inferior_set_breakpoint(inferior: TrapInferior, location: u64) -> TrapBreakpoint {
    let aligned_address = location & !0x7u64;
    let user_bp = Box::new(Breakpoint {
        shift: (location - aligned_address) * 8,
        aligned_address: InferiorPointer(aligned_address),
        target_address: InferiorPointer(location),
        original_breakpoint_word: peek_text(inferior, InferiorPointer(aligned_address)),
    });

    set(inferior, *user_bp);

    user_bp
}

Let me fix all the other uses. Hmm, how do I implement find_breakpoint_matching_inferior_instruction_pointer? It is called when all we know is that the inferior stopped somewhere and want to find the corresponding breakpoint.

We do know the inferior so maybe the inferior should hold on to breakpoints.
- But how do we remove the reference from the inferior?

Ok, just trying to push through incremental changes is not working well. Let’s take a step back and do some systems thinking about ownership and users.

Breakpoints:

Who uses the them / who needs to access the it?
- User needs it to determine when its hit.
  - On the other hand this comparing breakpoints in the callback is kind of gross. The user is passed the breakpoint in the callback.
    - Is comparing against the object something a client needs to do or just an artifact of the testing? This is also inherited from the C version that uses handles.
    - Worst case the callback could compare against the addresses.
- The breakpoint module needs access to the breakpoint to handle it properly.
Who should own it? Or who should control it’s lifetime?
- The user should decide if it is set or deactivated for flow control.
  - In order to deactivate (and also to deallocate) the user would need a way to refer to a breakpoint.
    - When creating the breakpoint with trap_inferior_set_breakpoint the user pases the address to refer to the breakpoint.
- Should the user decide if it is deallocated? Should it ever be deallocated?
  - If we have to search for a breakpoint then never deallocating them means a larger search space. Then again if we don’t need an index as a handle we can use a different datastructure, like a map.

Ok, that was a good exploration of the requirements. And note I used a question and answer investigation to work through it. I think the takeaway is:

The user doesn’t really need to own the breakpoint. The user certainly doesn’t need to look inside the Breakpoint struct.
The user does need a way to refer to breakpoints and the target_address might be a good one.
The user should control if a breakpoint is active or not but doesn’t necessarily need to control the lifetime.
It isn’t clear that deleting breakpoints is really useful for lifetime of the inferior.
The breakpoint module does need access to the breakpoint and the internals of the Breakpoint structure.

Rust’s ownership model offers surprising clarity when leaned into fully.

One thing I could do now is to create tests for all these behaviors and then enforce them. However, we still have a failing test and need to get past that first.

A design that these requirements seems to suggest is:

Store the breakpoints in a data structure attached to the inferior.
The data structure should be a map that maps target_address to Breakpoint.
The user can just refer to breakpoints by target address.
breakpoint::handle can pass the target_address to the callback and the tests can compare against the addresses rather than comparing breakpoints or breakpoint handles.

Ok, so how to do we get from where we are with the failing test to this new design? And using something like behavior driven development would be nice.

Approaching the Problem With a Clear Mind

I took my dogs for a walk and thought about this problem. This is a good way to clear my head and remove distractions, at least some kinds of distractions.

The key is getting past this failing test. I’ve only called that out like 3 times now.
I should do something to get the test to pass by brute force just so we can make progress and then clean up later.
This will give us passing tests we can use to validate our work step by step.
This will also let us write new tests for additional features.

To get the test passing by brute force I first thought of trying run each test as a separate run of the test program. This would acheive the goals but then I realized there is another thing we can try. I can try to reset GLOBAL_BREAKPOINTS between tests. Whenever a new inferior is created I could reset the breakpoints. From a code standpoint this will be messy, inferior has to reach into breakpoint to reset, but for the moment that’s fine. And we should have this fixed as we implement our new design. So let’s get started breaking through this failing test.

As I started to work on this I realized there is a simpler solution. If we return target_address as the breakpoint handle then it should also make the tests pass. We will still end up creating multiple breakpoints with the same target address but from User’s perspective it won’t matter. It only sees the target_address. This is a mess internally but it’s enough to break through this log jam and get us working. It’s also a step toward our new design in that we do want to use the target_address as the breakpoint identifier.

Ok, that does it. Here are the changes to the breakpoint module which just return the new TrapBreakpoint type:

modified   src/breakpoint/mod.rs
@@ -14,7 +14,7 @@ struct Breakpoint {
     original_breakpoint_word: i64,
 }
 
-pub type TrapBreakpoint = usize;
+pub type TrapBreakpoint = InferiorPointer;
 
 static mut GLOBAL_BREAKPOINTS: Vec<Breakpoint> = Vec::<Breakpoint>::new();
 
@@ -33,9 +33,9 @@ fn set(inferior: TrapInferior, bp: Breakpoint) {
 
 fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<(TrapBreakpoint, Breakpoint)> {
     let ip = ptrace_util::get_instruction_pointer(inf.pid).as_i64();
-    unsafe { for (idx, bp) in GLOBAL_BREAKPOINTS.iter().enumerate() {
+    unsafe { for bp in &GLOBAL_BREAKPOINTS {
 	if bp.target_address.as_i64() == ip - 1 {
-	    return Some((idx, *bp))
+	    return Some((bp.target_address, *bp))
 	}
     } };
     None
@@ -76,7 +76,7 @@ where
 
 pub fn trap_inferior_set_breakpoint(inferior: TrapInferior, location: u64) -> TrapBreakpoint {
     let aligned_address = location & !0x7u64;
-    let user_bp = unsafe {
+    let index = unsafe {
 	GLOBAL_BREAKPOINTS.push(Breakpoint {
             shift: (location - aligned_address) * 8,
             aligned_address: InferiorPointer(aligned_address),
@@ -86,7 +86,7 @@ pub fn trap_inferior_set_breakpoint(inferior: TrapInferior, location: u64) -> Tr
 	GLOBAL_BREAKPOINTS.len() - 1
     };
 
-    set(inferior, unsafe {GLOBAL_BREAKPOINTS[user_bp]});
+    set(inferior, unsafe {GLOBAL_BREAKPOINTS[index]});
 
-    user_bp
+    InferiorPointer(location)
 }

And I needed a few changes to make InferiorPointer a little more usable. I would actually like it if the tests could pass these in but I’ll need a little more work to support that.

modified   src/inferior/mod.rs
@@ -1,6 +1,7 @@
 use libc::c_void;
 use libc::pid_t;
 use std::ops::{Add, Sub};
+use std::fmt;
 
 #[derive(Copy, Clone)]
 pub enum InferiorState {
@@ -17,7 +18,7 @@ pub struct Inferior {
 
 pub type TrapInferior = pid_t;
 
-#[derive(Copy, Clone)]
+#[derive(Copy, Clone, PartialEq, PartialOrd, Debug)]
 pub struct InferiorPointer(pub u64);
 impl InferiorPointer {
     pub fn as_voidptr(&self) -> *mut c_void {
@@ -52,3 +53,9 @@ impl Sub<i64> for InferiorPointer {
         }
     }
 }
+impl fmt::Display for InferiorPointer {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        let &InferiorPointer(u) = self;
+	write!(f, "{}", u)
+    }
+}

I’ve implemented traits for PartialEq and PartialOrd so I can compare InferiorPointer structs and Display to print them out.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

With this the tests pass. There are warnings that we will want to clean up, but they pass. Let’s keep score on the warnings so we can see if we make progress on them along the way (we should as many are related to GLOBAL_BREAKPOINTS):

warning: `rusty_trap` (lib) generated 7 warnings
warning: `rusty_trap` (lib test) generated 7 warnings (7 duplicates)

...

running 4 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_handle_more_than_one_breakpoint ... ok
test it_can_set_breakpoints ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

From here we can start implementing the new design, recall that we want to:

Store the breakpoints in a data structure attached to the inferior.
The data structure should be a map that maps target_address to Breakpoint.
The user can just refer to breakpoints by target address.
breakpoint::handle can pass the target_address to the callback and the tests can compare against the addresses rather than comparing breakpoints or breakpoint handles.

And we’ve already completed “The user can just refer to breakpoints by target address” and “breakpoint::handle can pass the target_address to the callback”. Though there are some simplifications we can make. find_breakpoint_matching_inferior_instruction_pointer doesn’t need to return Option<(TrapBreakpoint, Breakpoint)> since TrapBreakpoint is now Breakpoint::target_address. I’ll make that change now.

modified   src/breakpoint/mod.rs
@@ -31,11 +31,11 @@ fn set(inferior: TrapInferior, bp: Breakpoint) {
     poke_text(inferior, bp.aligned_address, modified);
 }
 
-fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<(TrapBreakpoint, Breakpoint)> {
+fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<Breakpoint> {
     let ip = ptrace_util::get_instruction_pointer(inf.pid).as_i64();
     unsafe { for bp in &GLOBAL_BREAKPOINTS {
 	if bp.target_address.as_i64() == ip - 1 {
-	    return Some((bp.target_address, *bp))
+	    return Some(*bp)
 	}
     } };
     None
@@ -46,7 +46,7 @@ where
     F: FnMut(TrapInferior, TrapBreakpoint),
 {
     let inferior = inf.pid;
-    let (user_breakpoint, bp) =
+    let bp =
         find_breakpoint_matching_inferior_instruction_pointer(inf)
         .expect("Could not find breakpoint");
 
@@ -54,7 +54,7 @@ where
         InferiorState::Running => (),
         _ => panic!("Unhandled error in breakpoint::handle"),
     }
-    callback(inferior, user_breakpoint);
+    callback(inferior, bp.target_address);
     step_over(inferior, bp);
     return match waitpid(Pid::from_raw(inf.pid), None) {
         Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {

Ok, and that still passes and still has 7 warnings but these two design items are completely done.

Next, let’s add a map to the inferior. I started simply with this:

#[derive(Copy, Clone)]
pub struct Inferior {
    pub pid: pid_t,
    pub state: InferiorState,
    pub breakpoints: HashMap<InferiorPointer, Breakpoint>  // <== Added this
}

but this doesn’t build because I’ve dervied the Copy trait:

error[E0204]: the trait `Copy` cannot be implemented for this type
  --> src/inferior/mod.rs:16:10
   |
16 | #[derive(Copy, Clone)]
   |          ^^^^
...
20 |     pub breakpoints: HashMap<InferiorPointer, Breakpoint>
   |     ----------------------------------------------------- this field does not implement `Copy`

This makes sense, we don’t want to copy the map. So I should remove Copy, what about Clone?

Differs from Copy in that Copy is implicit and an inexpensive bit-wise copy, while Clone is always explicit and may or may not be expensive. [1]

Right, this is the deep copy. I guess there is no harm in deriving this. I’ll remove Copy and see what the fallout is. And it begins here:

error[E0382]: use of moved value: `inf`
  --> src/breakpoint/mod.rs:69:25
   |
44 | pub fn handle<F>(inf: Inferior, callback: &mut F) -> InferiorState
   |                  --- move occurs because `inf` has type `inferior::Inferior`, which does not implement the `Copy` trait
...
50 |         find_breakpoint_matching_inferior_instruction_pointer(inf)
   |                                                               --- value moved here
...
69 |                 signal, inf.state as i32
   |                         ^^^^^^^^^ value used here after move
   |
note: consider changing this parameter type in function `find_breakpoint_matching_inferior_instruction_pointer` to borrow instead if owning the value isn't necessary
   |
34 | fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<B...
   |    -----------------------------------------------------      ^^^^^^^^ this parameter takes ownership of the value
   |    |
   |    in this function
help: consider cloning the value if the performance cost is acceptable
   |
50 |         find_breakpoint_matching_inferior_instruction_pointer(inf.clone())
   |                                                                  ++++++++

Ok, so find_breakpoint_matching_inferior_instruction_pointer can either borrow or I can pass a clone. find_breakpoint_matching_inferior_instruction_pointer doesn’t need to own the inferior it just needs to use it to look up the instruction pointer. Oh, it will use the new breakpoints hashmap as well. So, it should borrow, immutably. Ok, that’s easy enough:

modified   src/breakpoint/mod.rs
@@ -31,7 +31,7 @@ fn set(inferior: TrapInferior, bp: Breakpoint) {
     poke_text(inferior, bp.aligned_address, modified);
 }
 
-fn find_breakpoint_matching_inferior_instruction_pointer(inf: Inferior) -> Option<Breakpoint> {
+fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<Breakpoint> {
     let ip = ptrace_util::get_instruction_pointer(inf.pid).as_i64();
     unsafe { for bp in &GLOBAL_BREAKPOINTS {
 	if bp.target_address.as_i64() == ip - 1 {

And then fix the call site as well.

Then we have another similar error:

error[E0382]: assign to part of moved value: `inf`
  --> src/lib.rs:88:9
   |
85 |       let mut inf = unsafe { global_inferior };
   |           ------- move occurs because `inf` has type `inferior::Inferior`, which does not implement the `Copy` trait
86 |       ptrace_util::cont(inf.pid);
87 |       loop {
   |       ---- inside of this loop
88 | /         inf.state = match waitpid(Pid::from_raw(inf.pid), None) {
89 | |             Ok(WaitStatus::Exited(_pid, code)) => return code,
90 | |             Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => breakpoint::handle(inf, c...
   | |                                                                                  --- value moved here
91 | |             Ok(WaitStatus::Stopped(_pid, signal)) => {
...  |
98 | |             Err(_) => panic!("Unhandled error in trap_inferior_continue"),
99 | |         };
   | |_________^ value partially assigned here after move
   |
note: consider changing this parameter type in function `handle` to borrow instead if owning the value isn't necessary
  --> src/breakpoint/mod.rs:44:23
   |
44 | pub fn handle<F>(inf: Inferior, callback: &mut F) -> InferiorState
   |        ------         ^^^^^^^^ this parameter takes ownership of the value
   |        |
   |        in this function
help: consider cloning the value if the performance cost is acceptable
   |
90 |             Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => breakpoint::handle(inf.clone(), callback),
   |                                                                                     ++++++++

Ah, ok this is dealing with the global_inferior. One option is to just compute the state and then unsafe assign into the global_inferior. But I think we will be getting rid of global_inferior soon because there are a number of other errors and warnings related to it. The main set of issues being related to initializing global_inferior in the first place. I’ve added this line to try to initialize breakpoints but it fails:

@@ -26,6 +27,7 @@ use breakpoint::TrapBreakpoint;
 static mut global_inferior: Inferior = Inferior {
     pid: 0,
     state: InferiorState::Stopped,
+    breakpoints: HashMap::new(),
 };

error[E0015]: cannot call non-const associated function `HashMap::<inferior::InferiorPointer, Breakpoint>::new` in statics
  --> src/lib.rs:30:18
   |
30 |     breakpoints: HashMap::new(),
   |                  ^^^^^^^^^^^^^^
   |
   = note: calls in statics are limited to constant functions, tuple structs and tuple variants
   = note: consider wrapping this expression in `std::sync::LazyLock::new(|| ...)`

And also here, I haven’t initialized breakpoints but don’t expect it to go well:

error[E0063]: missing field `breakpoints` in initializer of `inferior::Inferior`
  --> src/lib.rs:58:69
   |
58 |         Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior {
   |                                                                     ^^^^^^^^ missing `breakpoints`

Well, maybe this one would be easier to initialize since it isn’t static global. But, I think the right option here is to dynamically allocate the inferior and return it (and its ownership) to the user. Then borrow it from them during these operations. Hmm, is that right?

If the user passes it into trap_inferior_continue then rusty_trap is borrowing it from them.
When rusty_trap calls the callback it lends it back to the user.
So the user won’t be able to reference their original copy.
The tests won’t be able to write: assert_eq!(passed_inferior, inferior);
- Instead it could compare the process ids.
Would a real app want to compare?
- I guess it would as that’s the only way for it to know which breakpoint is which.

Note, all of this depends on the app needing more than one inferior which might be rare but is an intended feature of the debugger. This gets into what the purpose of a debugger like this is for. I’ll come back to that at the end of the post.

But let’s move forward with dynamically allocating the inferior. First, a function, new, to intialize the Inferior.

#[derive(Copy, Clone, PartialEq, PartialOrd, Debug)]
pub struct InferiorPointer(pub u64);
impl InferiorPointer {
    // ...
    pub fn new(pid: pid_t) -> Inferior {
	    Inferior {
			pid,
			state: InferiorState::Stopped,
			breakpoints: HashMap::new(),
		}
    }
}

Then the rest:

modified   src/breakpoint/mod.rs
@@ -18,17 +18,17 @@ pub type TrapBreakpoint = InferiorPointer;
 
 static mut GLOBAL_BREAKPOINTS: Vec<Breakpoint> = Vec::<Breakpoint>::new();
 
-fn step_over(inferior: TrapInferior, bp: Breakpoint) {
-    poke_text(inferior, bp.aligned_address, bp.original_breakpoint_word);
-    set_instruction_pointer(inferior, bp.target_address);
-    single_step(inferior);
+fn step_over(inferior: &TrapInferior, bp: Breakpoint) {
+    poke_text(inferior.pid, bp.aligned_address, bp.original_breakpoint_word);
+    set_instruction_pointer(inferior.pid, bp.target_address);
+    single_step(inferior.pid);
 }
 
-fn set(inferior: TrapInferior, bp: Breakpoint) {
+fn set(inferior: &TrapInferior, bp: Breakpoint) {
     let mut modified = bp.original_breakpoint_word;
     modified &= !(0xFFi64 << bp.shift);
     modified |= 0xCCi64 << bp.shift;
-    poke_text(inferior, bp.aligned_address, modified);
+    poke_text(inferior.pid, bp.aligned_address, modified);
 }
 
 fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<Breakpoint> {
@@ -41,32 +41,31 @@ fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Opti
     None
 }
 
-pub fn handle<F>(inf: Inferior, callback: &mut F) -> InferiorState
+pub fn handle<F>(inferior: &mut Inferior, callback: &mut F) -> InferiorState
 where
-    F: FnMut(TrapInferior, TrapBreakpoint),
+    F: FnMut(&TrapInferior, TrapBreakpoint),
 {
-    let inferior = inf.pid;
     let bp =
-        find_breakpoint_matching_inferior_instruction_pointer(&inf)
+        find_breakpoint_matching_inferior_instruction_pointer(inferior)
         .expect("Could not find breakpoint");
 
-    match inf.state {
+    match inferior.state {
         InferiorState::Running => (),
         _ => panic!("Unhandled error in breakpoint::handle"),
     }
     callback(inferior, bp.target_address);
     step_over(inferior, bp);
-    return match waitpid(Pid::from_raw(inf.pid), None) {
+    return match waitpid(Pid::from_raw(inferior.pid), None) {
         Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => {
             set(inferior, bp);
-            cont(inferior);
+            cont(inferior.pid);
             InferiorState::Running
         }
         Ok(WaitStatus::Exited(_pid, _code)) => InferiorState::Running,
         Ok(WaitStatus::Stopped(_pid, signal)) => {
             panic!(
                 "Unexpected stop on signal {} in breakpoint::handle.  State: {}",
-                signal, inf.state as i32
+                signal, inferior.state as i32
             )
         }
         Ok(_) => panic!("Unexpected stop in breakpoint::handle"),
@@ -74,14 +73,14 @@ where
     };
 }
 
-pub fn trap_inferior_set_breakpoint(inferior: TrapInferior, location: u64) -> TrapBreakpoint {
+pub fn trap_inferior_set_breakpoint(inferior: &TrapInferior, location: u64) -> TrapBreakpoint {
     let aligned_address = location & !0x7u64;
     let index = unsafe {
 	GLOBAL_BREAKPOINTS.push(Breakpoint {
             shift: (location - aligned_address) * 8,
             aligned_address: InferiorPointer(aligned_address),
             target_address: InferiorPointer(location),
-            original_breakpoint_word: peek_text(inferior, InferiorPointer(aligned_address)),
+            original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address)),
 	});
 	GLOBAL_BREAKPOINTS.len() - 1
     };
modified   src/inferior/mod.rs
@@ -17,10 +17,20 @@ pub enum InferiorState {
 pub struct Inferior {
     pub pid: pid_t,
     pub state: InferiorState,
-    pub breakpoints: HashMap<InferiorPointer, Breakpoint>  // Added this
+    pub breakpoints: HashMap<InferiorPointer, Breakpoint>
 }
 
-pub type TrapInferior = pid_t;
+impl Inferior {
+    pub fn new(pid: pid_t) -> Inferior {
+	Inferior {
+	    pid,
+	    state: InferiorState::Stopped,
+	    breakpoints: HashMap::new(),
+	}
+    }
+}
+
+pub type TrapInferior = Inferior;
 
 #[derive(Copy, Clone, PartialEq, PartialOrd, Debug)]
 pub struct InferiorPointer(pub u64);
modified   src/lib.rs
@@ -10,7 +10,6 @@ use nix::{
     unistd::{execve, fork, ForkResult},
     Error,
 };
-use std::collections::HashMap;
 use std::ffi::{CStr, CString};
 use std::path::Path;
 
@@ -24,12 +23,6 @@ mod breakpoint;
 pub use self::breakpoint::trap_inferior_set_breakpoint;
 use breakpoint::TrapBreakpoint;
 
-static mut global_inferior: Inferior = Inferior {
-    pid: 0,
-    state: InferiorState::Stopped,
-    breakpoints: HashMap::new(),
-};
-
 fn disable_address_space_layout_randomization() {
     unsafe {
         let old = libc::personality(0xffffffff);
@@ -55,49 +48,46 @@ fn exec_inferior(filename: &Path, args: &[&str]) {
 fn attach_inferior(raw_pid: pid_t) -> Result<Inferior, Error> {
     let nix_pid = Pid::from_raw(raw_pid);
     match waitpid(nix_pid, None) {
-        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior {
-            pid: pid.into(),
-            state: InferiorState::Running,
-        }),
+        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) => Ok(Inferior::new(pid.into())),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => Err(e),
     }
 }
 
-pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior, Error> {
+pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<Inferior, Error> {
     loop {
         match unsafe { fork() } {
-            Ok(ForkResult::Child) => exec_inferior(filename, args),
+            Ok(ForkResult::Child) => {
+		exec_inferior(filename, args);
+		unreachable!();
+	    }
             Ok(ForkResult::Parent { child: pid }) => {
-                unsafe { global_inferior = attach_inferior(pid.into()).ok().unwrap() };
-                return Ok(pid.into());
-            }
+		return attach_inferior(pid.into())
+	    }
             Err(Error::EAGAIN) => continue,
             Err(e) => return Err(e),
         }
     }
 }
 
-pub fn trap_inferior_continue<F>(inferior: TrapInferior, callback: &mut F) -> i32
+pub fn trap_inferior_continue<F>(inferior: &mut TrapInferior, callback: &mut F) -> i32
 where
-    F: FnMut(TrapInferior, TrapBreakpoint),
+    F: FnMut(&TrapInferior, TrapBreakpoint),
 {
-    let mut inf = unsafe { global_inferior };
-    ptrace_util::cont(inf.pid);
+    ptrace_util::cont(inferior.pid);
     loop {
-        inf.state = match waitpid(Pid::from_raw(inf.pid), None) {
+        inferior.state = match waitpid(Pid::from_raw(inferior.pid), None) {
             Ok(WaitStatus::Exited(_pid, code)) => return code,
-            Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) => breakpoint::handle(inf, callback),
+            Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) =>
+		breakpoint::handle(inferior, callback),
             Ok(WaitStatus::Stopped(_pid, signal)) => {
                 panic!(
                     "Unexpected stop on signal {} in trap_inferior_continue.  State: {}",
-                    signal, inf.state as i32
+                    signal, inferior.state as i32
                 )
             }
             Ok(_) => panic!("Unexpected stop in trap_inferior_continue"),
             Err(_) => panic!("Unhandled error in trap_inferior_continue"),
         };
-
-        unsafe { global_inferior = inf };
     }
 }

And then we have the problem I expected that the tests can’t reference the inferior because they lent it to trap_inferior_continue:

error[E0502]: cannot borrow `inferior` as immutable because it is also borrowed as mutable
  --> tests/lib.rs:24:60
   |
24 |     rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
   |     ---------------------------------- -------------       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ immutable borrow occurs here
   |     |                                  |
   |     |                                  mutable borrow occurs here
   |     mutable borrow later used by call
25 |         assert_eq!(passed_inferior.pid, inferior.pid);
   |                                         ------------ second borrow occurs due to use of `inferior` in closure

Ok, good well at least I’m understanding Rust well enough to have expected this. For now, the test will need to copy out the pid before lending the inferior back to rusty_trap

And with that, the tests pass. And, we are down to only 4 warnings!

warning: unused variable: `args`
  --> src/lib.rs:33:35
   |
33 | fn exec_inferior(filename: &Path, args: &[&str]) {
   |                                   ^^^^ help: if this is intentional, prefix it with an underscore: `_args`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: creating a shared reference to mutable static
  --> src/breakpoint/mod.rs:36:24
   |
36 |     unsafe { for bp in &GLOBAL_BREAKPOINTS {
   |                        ^^^^^^^^^^^^^^^^^^^ shared reference to mutable static
   |
   = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/static-mut-references.html>
   = note: shared references to mutable statics are dangerous; it's undefined behavior if the static is mutated or if a mutable reference is created for it while the shared reference lives
   = note: `#[warn(static_mut_refs)]` on by default
help: use `&raw const` instead to create a raw pointer
   |
36 |     unsafe { for bp in &raw const GLOBAL_BREAKPOINTS {
   |                         +++++++++

warning: creating a mutable reference to mutable static
  --> src/breakpoint/mod.rs:79:2
   |
79 | /     GLOBAL_BREAKPOINTS.push(Breakpoint {
80 | |             shift: (location - aligned_address) * 8,
81 | |             aligned_address: InferiorPointer(aligned_address),
82 | |             target_address: InferiorPointer(location),
83 | |             original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address)),
84 | |     });
   | |______^ mutable reference to mutable static
   |
   = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/static-mut-references.html>
   = note: mutable references to mutable statics are dangerous; it's undefined behavior if any other pointer to the static is used or if any other reference is created for the static while the mutable reference lives

warning: creating a shared reference to mutable static
  --> src/breakpoint/mod.rs:85:2
   |
85 |     GLOBAL_BREAKPOINTS.len() - 1
   |     ^^^^^^^^^^^^^^^^^^^^^^^^ shared reference to mutable static
   |
   = note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2024/static-mut-references.html>
   = note: shared references to mutable statics are dangerous; it's undefined behavior if the static is mutated or if a mutable reference is created for it while the shared reference lives

warning: `rusty_trap` (lib) generated 4 warnings
warning: `rusty_trap` (lib test) generated 4 warnings (4 duplicates)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.36s
     Running unittests src/lib.rs (/home/jkain/projects/rusty_trap/target/debug/deps/rusty_trap-dd302d51f38b1a8c)

running 4 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... ok
test it_can_handle_more_than_one_breakpoint ... ok
test it_can_set_breakpoints ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Ok, so these warnings all about GLOBAL_BREAKPOINTS. We have added the breakpoints HashMap to the inferior but aren’t using it yet. Let’s do that now.

The main change is to update find_breakpoint_matching_inferior_instruction_pointer like this:

-fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<Breakpoint> {
-    let ip = ptrace_util::get_instruction_pointer(inf.pid).as_i64();
-    unsafe { for bp in &GLOBAL_BREAKPOINTS {
-	if bp.target_address.as_i64() == ip - 1 {
-	    return Some(*bp)
-	}
-    } };
-    None
+fn find_breakpoint_matching_inferior_instruction_pointer(inf: &Inferior) -> Option<&Breakpoint> {
+    let InferiorPointer(ip) = get_instruction_pointer(inf.pid);
+    let ip = InferiorPointer(ip - 1);
+    return inf.breakpoints.get(&ip);
 }

Then just some minor changes to change Breakpoint to &Breakpoint, etc. And finally, we remove GLOBAL_BREAKPOINTS from everything else:

modified   src/breakpoint/mod.rs
@@ -67,19 +65,17 @@ where
     };
 }
 
-pub fn trap_inferior_set_breakpoint(inferior: &TrapInferior, location: u64) -> TrapBreakpoint {
+pub fn trap_inferior_set_breakpoint(inferior: &mut TrapInferior, location: u64) -> TrapBreakpoint {
     let aligned_address = location & !0x7u64;
-    let index = unsafe {
-	GLOBAL_BREAKPOINTS.push(Breakpoint {
-            shift: (location - aligned_address) * 8,
-            aligned_address: InferiorPointer(aligned_address),
-            target_address: InferiorPointer(location),
-            original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address)),
-	});
-	GLOBAL_BREAKPOINTS.len() - 1
-    };
+    let target_address = InferiorPointer(location);
+    inferior.breakpoints.insert(target_address, Breakpoint {
+        shift: (location - aligned_address) * 8,
+        aligned_address: InferiorPointer(aligned_address),
+        target_address,
+        original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address)),
+    });
 
-    set(inferior, unsafe {GLOBAL_BREAKPOINTS[index]});
+    set(inferior, inferior.breakpoints.get(&target_address).unwrap());
 
     InferiorPointer(location)
 }
modified   tests/lib.rs
@@ -21,7 +21,7 @@ fn it_can_set_breakpoints() {
     let mut inferior =
         rusty_trap::trap_inferior_exec(Path::new("./target/debug/twelve"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp = rusty_trap::trap_inferior_set_breakpoint(&inferior, 0x000055555555b821);
+    let bp = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, 0x000055555555b821);
     rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
@@ -38,7 +38,7 @@ fn it_can_handle_a_breakpoint_more_than_once() {
     let mut inferior =
         rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp = rusty_trap::trap_inferior_set_breakpoint(&inferior, ADDRESS_OF_FOO);
+    let bp = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_FOO);
     rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         assert_eq!(passed_bp, bp);
@@ -56,8 +56,8 @@ fn it_can_handle_more_than_one_breakpoint() {
     let mut inferior =
         rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[]).unwrap();
     let expected_pid = inferior.pid;
-    let bp_main = rusty_trap::trap_inferior_set_breakpoint(&inferior, ADDRESS_OF_MAIN);
-    let bp_foo = rusty_trap::trap_inferior_set_breakpoint(&inferior, ADDRESS_OF_FOO);
+    let bp_main = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_MAIN);
+    let bp_foo = rusty_trap::trap_inferior_set_breakpoint(&mut inferior, ADDRESS_OF_FOO);
     rusty_trap::trap_inferior_continue(&mut inferior, &mut |passed_inferior, passed_bp| {
         assert_eq!(passed_inferior.pid, expected_pid);
         if passed_bp == bp_main {

And the tests pass and there is only one warning left:

warning: unused variable: `args`
  --> src/lib.rs:33:35
   |
33 | fn exec_inferior(filename: &Path, args: &[&str]) {
   |                                   ^^^^ help: if this is intentional, prefix it with an underscore: `_args`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: `rusty_trap` (lib) generated 1 warning

Ok, I guess this was a placeholder I left in case the user wants to pass arguments to the inferior. In general this should be supported but I’m not going to build it and it’s not really that interesting for this blog. So, I’m just going to take the compiler’s advice and rename the argument _args_. If you have an interest in this please go ahead and implement it.

And now, it’s clean. And not only that, the tests pass without the --test-threads 1 option. Nice.

Aside on API design?

Now, I say it’s clean meaning it compiled without any warnings. But I don’t really like the API, it doesn’t feel clean to me. For you rustaceans that have a better knowledge of idiomatic Rust than I do, what’s the right way to create APIs like trap_inferior_exec? I’m dissatisfied with the way it is now because it forces the user to have a mutable Inferior just so that it can pass it back into other rusty_trap functions. The user doesn’t really want to know what’s in there or modify anything. They would depend on rusty_trap to do that for them. So I’d like to return at the very least an immutable object. I guess it’s mutable because they expect rusty_trap to mutate it. Let me know in the comments what you think about this or if you have an suggestions for a cleaner API.

But for now I’ve pushed and merged a PR:

Conclusion: From Global State to Ownership

In this post, we’ve seen how implementing multiple breakpoints in Rust pushed us to reconsider our design approach fundamentally. What started as a seemingly straightforward port from C evolved into an exploration of Rust’s ownership principles and safer design patterns.

Throughout this journey, I applied the question and answer investigation approach to systematically break down problems:

Breakpoints:

Who uses them / who needs to access them?

Who should own them? Or who should control their lifetime?

Should the user decide if it is deallocated? Should it ever be deallocated?

This structured questioning helped clarify requirements and guide us toward better solutions. When I hit roadblocks, I asked specific questions about ownership relationships and data flow, letting the answers drive implementation decisions.

Our journey took us from:

A design with problematic global state and unsafe code
Through a failing test that exposed the limitations of that approach
To a cleaner design with explicit ownership of breakpoints tied to their inferior

The key lessons we learned include:

Global state is particularly problematic in Rust, which actively encourages you to make relationships explicit through the type system
Rust’s ownership model guides you toward designs where data relationships are clear and statically verified.
Even temporary “brute force” solutions can help you make incremental progress toward a better design
Asking the right questions about ownership and responsibility drives better architecture decisions

While our current API still has room for improvement, we’ve eliminated most of the unsafe code and made the relationship between inferiors and their breakpoints much more explicit. The tests now pass reliably without special handling for concurrency issues - a testament to the benefits of thoughful ownership.

What’s Next?

Looking ahead, there are several opportunities to improve our debugger:

Symbol tables: We’ll need to parse the inferiors symbol table to make function names to addresses rather than the fragile hard coding of addresses that we have now.
Dynamic library support: Using the debugger hook for loading and debugging dynamic libraries.
Multiple threads: Adding support for debugging multi-threaded applications.

Symbol tables will likely be next major topic, as they’re essential for providing a usable debugging experience. The Rust ecosystem’s strong support for parsing binary formats should make this an interesting and productive exploration.

Through this series, we’re not just learning how to build a debugger – we’re discovering how Rust’s design principles can guide us toward more robust software architecture decisions. The ownership-driven design we’ve developed here will continue to evolve as we add more complex debugging capabilities.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

How I read new code

Mon, 04 Aug 2025 00:00:00 +0000

Ring Theory

When I was in college I studied computer science but I also took a lot of math classes including abstract math. My intro to abstract algebra went well and I enjoyed the class. I decided to take the next course which was on ring theory, if I remember correctly. The beginning of this class started off as sort of a shock, yes it got more abstract and harder to understand but our professor also had an interesting approach. He assigned reading, we were to go back and read the text book and then do some exercises. And he didn’t really want to teach us the material via lecture, he wanted to have a discussion about the material in class. Ok, this seemed more like what I would expect in a philosophy or literature class but in math? I was really conditioned to learning math from lectures.

My friend Rob was in the class as well and we would study together. He and I tried to read the text. Our professor had told us to read until the point we lose understanding and then come to office hours and ask him for help. The problem was that we didn’t really have a single point where we would lose understanding. It was more that we had a gradual decay of understanding over time. Until at some point you realize you haven’t understood a thing for several paragraphs.

As our frustration with our lack of progress grew Rob and I decided to try something different. We flipped to the end of the chapter and started reading the exercises we had been assigned. We thought we should try to get through them anyway we could. This ended up working extremely well. I can’t remember any of the exercises we worked on that day but here’s one I found online in Ring Theory that can serve as an example:

6.1
1. Suppose that R satisfies all ring axioms for a ring with identity, except that for all
 xy in R, x+y=y+x. Show that this axiom is implied by the rest.

So we read the question and then asked ourselves, “what does this even mean?”

And we went back through the text to read and to find answers to the questions:

What is a ring?
What are the axioms for a ring with identity?

Then we at least understood the question. From there we could work with the other axioms to try to prove that for all xy in R, x+y=y+x.

Reading Code

Ok, so what does this story about my ring theory class have to do with reading code? Well, everything. I think the take away from that story is that:

The key to our understanding was the context created by the question in the exercise.

And this is maybe one of the most impactful things I’ve ever learned in my life.

So, when I read code that’s new to me I like to have a question to provide context. Without a question I just read function after function and my understanding of the system as a whole just starts to degrade over time.

When I’m starting out with a new system I get a small bug or task to work on and then read the code in the context of that bug or task. This helps me focus on a few details at a time and how they relate to the bug I’m working on, and in turn to each other. A small bug to fix or task to work on is something that a company might give you as a new hire, but I always treat as a way frame learning the system rather than as just an assignment.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

Top down / Bottom up

Reading functions is what I would call building a bottom up understanding of a system. Functions are low-level details within the system and conceivably if you read and understood all of them you could build a complete understanding of the system. However, what I find hard to understand when just reading functions is how they fit together. Sure, you can get an idea of how a few of them fit together if you read them in the right order. But, calls between functions can be far apart in the code base making it harder to build this understanding. Creating a context around a bug builds, what feels to me like, a container that ties a few functions together and makes it easier to think about the relationships. Pushing to the extreme is what I think of at top down thinking. That is, puting the relationships first - what are the overarching activities or data flows that are in a system? These activities and data flows are implemented using the functions in the system but the functions themselves don’t always reveal what the over arching flows are.

I once started a new job at a company that was building a distributed system. I was hired to do performance work and so I started looking at one of the nodes, let’s call it node A, in the system. Others had told me to take a look at node A because they thought it had high latency and would be a good place to start. I dug into node A with the context of: where does this node do most of its work? I did some profiling and made a few minor improvements. Great, I got something accomplished and I learned a little more about node A.

From there I looked at another node and realized a couple of things:

The individual nodes were relatively easy to read.
There was a common structure, a base class, that all the nodes were built on that gave them a consistent structure and life cycle while providing pub/sub capability.

Understanding this structure helped me to fit other nodes into a mental model so that I could dig into them and understand them more quickly.

I realized that to really look at the latency of the system as a whole I needed to better understand the system as a whole. Looking at nodes randomly or based on rumor was only going to get me so far.

This is what I consider top down vs. bottom up thinking. Looking at individual nodes in the distributed system is bottom up. It’s great for understanding the details of a system. And in fact, for something like performance, a lot of improvements will come from making changes in the details. That’s where work gets done, in the details.

That said, a top down approach is also very useful and can lead to even more significant improvements in performance. Before I describe how, let me first expand on what I mean by top down. I think of the top down understanding of a system like this as understanding how the parts fit together and how information or work flows through the system. In the case of this distributed system, each node performs a different kind of task and they collaborate to solve a larger problem. How they are put together is critical.

The classical performance strategy for a system like this is to understand the critical path through the graph of nodes. To do this you need to understand the system, first of all where are the inputs? Where does data come into the system from the external world? This is the “start” of your critical path. Where do the outputs come from (and how do they get delivered to the external world)? This the end of the critical path. Then the critical path through the nodes in between can be identified and work can be done to improve it.

Another example from the system I worked on was that there was a lot of time spent “between” the nodes or in nodes deciding when to act. Some nodes took input from several other nodes and might have to decide to wait or timeout when data isn’t available from a specific input. Acting sooner reduces latency but risks missing newer inputs. We have to ask, which inputs are critical? Are there certain inputs that should be acted on immediately (e.g. for safety reasons). Are there certain inputs that can’t be dropped (also for safety reasons). If so, not waiting for other inputs can be the right solution here and can save time. But to decide this we need to understand the system holistically, to know what’s important and what’s critical to safety.

Earlier, I promised to explain how top down thinking can be used to make significant improvements in performance. Well, in my mind bottom up thinking is great at identifying where time is spend and then allowing you to make those activities faster thereby improving performance. But, working top down is the best way to identify work that doesn’t have to be done at all. By understanding the connections between the nodes we can identify redundant work or unimportant work and eliminate or deprioritize it so the rest of the system can focus on what’s important.

While both top down and bottom up understandings are valuable I tend to have a preference for top down understanding. Bottom up understanding is more concerned with details, like individual functions. I find that:

The functions aren’t that important on their own. It’s how they work together that really shows what’s going on.
In a system, there are a lot of more details than there are architectures or overarching structures so keeping the top down view in mind is a lot easier.

The problem I find with focusing on the details is that I find that they generally change more rapidly than the overall system architecture. So I tend to understand functions on demand – a bug or task takes me to a function or set of functions and I read it, understand it, work with, and move on. And, I often found when working in high paced companies that the next time I came back to the function it had changed, or been removed, or was used in a completely new way. So getting intimately knowledgeable about that function didn’t pay off. But understanding larger aspects of the system helped so that when I had a bug or task to work on I knew which part of the system to look at. The big blocks of functionality don’t change that frequently so I could come back to them with a broad understanding and come up to speed on the details quickly.

Wrapping Up

Don’t get me wrong, the details are important too.

The details are where the work happens. And the top down architecture or structure is where the decisions about what work to do are made.

I guess what I’m saying is, when diving into a new system: don’t sweat the small stuff.

Handling multiple breakpoints in Trap Redux

Mon, 28 Jul 2025 00:00:00 +0000

In a previous post I worked on getting Trap building on a new system and found that the unit tests were not passing. It looked like I had found a real bug:

I found:

The first call to breakpoint_resolve must be when the breakpoint in main is reached. And the second call is 3 bytes later. I think this is trying to resolve the stop in execution that happens when we are in state INFERIOR_SINGLE_STEPPING to step over the breakpoint. And in this case the instruction pointer will have moved past the byte where we inserted the breakpoint – past the entire original instruction. So breakpoint_resolve should fail as this address isn’t one of our breakpoints. That is, breakpoint_resolve assumes we executed a one byte instruction and subtracts 1 from the instruction pointer to find the target_address.

This is not correct and wasn’t correct when I originally wrote it. I don’t know how this worked before.

So, to make progress I’m going to go back to the commit I started with at the beginning of Handling multiple breakpoints in Trap to retrace my steps without introducing this bug. So I’m starting at commit fb7b3c6 . First, I need to cherry-pick my changes to fix the build and disable address space randomization.

modified   src/inferior_load.c
@@ -5,6 +5,8 @@
 #include <sys/wait.h>
 #include <assert.h>
 #include <errno.h>
+#include <sys/personality.h>
+#include <signal.h>
 #include <stdio.h>
 #include <stdlib.h>
 
@@ -12,6 +14,10 @@ static inferior_t g_inferior;
 
 static void setup_inferior(const char *path, char *const argv[])
 {
+  unsigned long old = personality(0xFFFFFFFF);
+  if (personality(old | ADDR_NO_RANDOMIZE) < 0) {
+    perror("Failed to set personality:");
+  }
   ptrace_util_traceme();
   execv(path, argv);
 }

Alright, it builds and the tests fail because I need to update the breakpoint addresses again. I’ll follow the style of the nice clean up that I used in rusty_trap that sets some constants for the two addresses I use (main and foo)

modified   test/test_breakpoint.c
@@ -2,6 +2,8 @@
 #include <trap.h>
 #include <unistd.h>
 
+const char *ADDRESS_OF_MAIN = (char*)0x0000555555555151;
+
 int g_breakpoint_count = 0;
 trap_inferior_t g_inferior = 0;
 trap_breakpoint_t g_bp = 0;
@@ -20,7 +22,7 @@ int main()
 
   trap_breakpoint_set_callback(breakpoint_callback);
   g_inferior = trap_inferior_exec("./inferiors/hello", argv);
-  g_bp = trap_inferior_set_breakpoint(g_inferior, (char *)0x000000000040079d);
+  g_bp = trap_inferior_set_breakpoint(g_inferior, ADDRESS_OF_MAIN);
   trap_inferior_continue(g_inferior);
 
   assert(g_breakpoint_count == 1);
modified   test/test_multi_breakpoint.c
@@ -2,6 +2,8 @@
 #include <trap.h>
 #include <unistd.h>
 
+const char *ADDRESS_OF_FOO = (char*)0x0000555555555131;
+
 int g_breakpoint_count = 0;
 trap_inferior_t g_inferior = 0;
 trap_breakpoint_t g_bp = 0;
@@ -20,7 +22,7 @@ int main()
 
   trap_breakpoint_set_callback(breakpoint_callback);
   g_inferior = trap_inferior_exec("./inferiors/loop", argv);
-  g_bp = trap_inferior_set_breakpoint(g_inferior, (char *)0x000000000040076d);
+  g_bp = trap_inferior_set_breakpoint(g_inferior, ADDRESS_OF_FOO);
   trap_inferior_continue(g_inferior);
 
   assert(g_breakpoint_count == 5);

There are two interesting things to note here.

The first is minor: I created const globals to store the address out of habit and now there is a warming that passing the constant to trap_inferior_set_breakpoint discards const. I should fix the signature. This can be const.

As for the second, why do I already have a test_multi_breakpoint?!?! I thought the point of Handling multiple breakpoints in Trap was to build that functionality? And strangely the test passes! Ok, let me review Handling multiple breakpoints in Trap again - the tagline does say I also worked on handling more than one breakpoint not just breakpoints triggered multiple times.

I asked Claude 3.7 Sonnet Thought to summarize the post for me. I like doing this because it can make sure I don’t miss anything. The one complaint I have is that no matter how much brevity I ask for it always gives me too much. Anyway, here’s the summary of the main steps:

AI Generated Summary

Initial Goals

Address warnings from previous state machine implementation
Write a test for multiple breakpoints
Make the test pass

Implementing Multiple Breakpoints Support

Created a new test (test_multiple_breakpoints.c) that sets two breakpoints
Identified the core issue: reliance on a single global g_breakpoint variable
Replaced global variable with an array of breakpoints (g_breakpoints[MAX_BREAKPOINTS])
Implemented breakpoint_allocate() to get breakpoints from the array

Resolving the Right Breakpoint

Implemented ptrace_util_get_instruction_pointer() to determine current position
Created breakpoint_resolve() to find which breakpoint was hit
Implemented find_breakpoint_with_target_address() for lookup

Handling Duplicate Breakpoints

Created a test that sets two breakpoints at the same address
Considered API design options (abort, return same handle, create distinct handles)
Decided on option to return the same handle for duplicate breakpoints

Removing Breakpoints

Added functionality to remove breakpoints (trap_inferior_remove_breakpoint)
Created tests for removing breakpoints (both before execution and during callbacks)
Discovered issues when removing breakpoints from within callbacks

Key Bug: Duplicate Breakpoint Reading

Identified that when setting a second breakpoint at the same address, the code was reading the modified memory (with INT 3 already inserted) rather than using the original instruction
Fixed by checking if a breakpoint already exists at an address and copying its original_breakpoint_word rather than reading memory

Deferred Removal Implementation

Added state tracking to breakpoints (BREAKPOINT_ACTIVE, BREAKPOINT_DEFERED_REMOVE)
Implemented deferred removal for breakpoints deleted during callbacks
Created do_deferred_removals() to process pending removals

Refactoring and Cleanup

Balanced structure between start_breakpoint() and finish_breakpoint()
Identified duplication in code that iterates over breakpoint collections

Final Issues

Noted need for more robust breakpoint container
Identified remaining warnings indicating incomplete support for multiple inferiors

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

My Summary of the Summary

So Handling multiple breakpoints in Trap was really about setting multiple breakpoints and not at all about handling a single breakpoint multiple times. The latter works now. Handling more than one breakpoint introduced the function breakpoint_resolve which is used when the inferior stops to look up which breakpoint was hit using the instruction pointer as a key. Then we went on to work through the details of handling the breakpoints, fixing bugs, and cleaning up.

Most of what I see in the summary sounds pretty reasonable with the exception of breakpoint_resolve. As we saw before it doesn’t work because each breakpoint trigger causes the inferior to stop twice:

When the breakpoint is executed - breakpoint_resolve works in this case.
When single stepping over the original instruction - breakpoint_resolve does not work in this case.

Retracing Our Steps

Given that most of what was done in Handling multiple breakpoints in Trap seems like the right things to do let’s retrace our steps using the summary above as a map and the old post for more detail. When we get to breakpoint_resolve we can take what we know now and come up with better test coverage and drive a more robust solution.

So, let’s start with

Created a new test (test_multiple_breakpoints.c) that sets two breakpoints

Let’s actually just take the original version of the test. For trap implementation code I’m not sure I want to just take the old versions. But for the tests I think I do. By using the same test we can drive toward the same feature set and API while building a more robust solution. Here’s the test:

#include <assert.h>
#include <trap.h>
#include <unistd.h>

int g_foo_count = 0;
int g_main_count = 0;

trap_inferior_t g_inferior = 0;
trap_breakpoint_t g_bp_main = 0;
trap_breakpoint_t g_bp_foo  = 0;

void breakpoint_callback(trap_inferior_t inferior, trap_breakpoint_t bp)
{
  assert(inferior == g_inferior);

  if (bp == g_bp_main) {
    g_main_count++;
  } else if (bp == g_bp_foo) {
    g_foo_count++;
  }
}

int main()
{
  char *argv[1] = { 0 };

  trap_breakpoint_set_callback(breakpoint_callback);
  g_inferior = trap_inferior_exec("./inferiors/loop", argv);
  g_bp_main = trap_inferior_set_breakpoint(g_inferior, (char *)0x0000000000400778);
  g_bp_foo  = trap_inferior_set_breakpoint(g_inferior, (char *)0x000000000040076d);
  trap_inferior_continue(g_inferior);

  assert(g_main_count == 1);
  assert(g_foo_count == 5);

  return 0;
}

I will update the addresses though and extract them into constants.

modified   test/test_multiple_breakpoints.c
@@ -2,6 +2,9 @@
 #include <trap.h>
 #include <unistd.h>
 
+const char *ADDRESS_OF_MAIN = (char*)0x0000555555555140;
+const char *ADDRESS_OF_FOO  = (char*)0x0000555555555131;
+
 int g_foo_count = 0;
 int g_main_count = 0;
 
@@ -26,8 +29,8 @@ int main()
 
   trap_breakpoint_set_callback(breakpoint_callback);
   g_inferior = trap_inferior_exec("./inferiors/loop", argv);
-  g_bp_main = trap_inferior_set_breakpoint(g_inferior, (char *)0x0000000000400778);
-  g_bp_foo  = trap_inferior_set_breakpoint(g_inferior, (char *)0x000000000040076d);
+  g_bp_main = trap_inferior_set_breakpoint(g_inferior, ADDRESS_OF_MAIN);
+  g_bp_foo  = trap_inferior_set_breakpoint(g_inferior, ADDRESS_OF_FOO);
   trap_inferior_continue(g_inferior);
 
   assert(g_main_count == 1);

And as expected this test fails because we use single global breakpoint and can’t track both the breakpoints the test uses. Let’s work on fixing that. Last time we:

Replaced global variable with an array of breakpoints (g_breakpoints[MAX_BREAKPOINTS])

Implemented breakpoint_allocate() to get breakpoints from the array

While I don’t think a global array of breakpoints is the ideal form let’s go in this direction for now. But, eventually I think we’ll need to dynamically allocate breakpoints.

Implementing Multiple Breakpoints with Working Resolution

Now we can start working on a proper implementation by writing breakpoint_allocate. Oh, now that I open up breakpoint.c and start looking for a place to add breakpoint_allocate I find:

static trap_breakpoint_t breakpoint_resolve(trap_inferior_t inferior)
{
  return &g_breakpoint;
}

So breakpoint_resolve has been here for a while. This old version ‘works’ because there is only one breakpoint so we can simple resolve to that single breakpoint. It doesn’t matter what state the inferior is in as there is only one breakpoint to consider. This function will need to do more to support multiple breakpoints.

Ok, at least adding an array of breakpoints and a simple allocator is straightforward:

modified   src/breakpoint.c
@@ -12,7 +12,8 @@ struct breakpoint_t {
 };
 typedef struct breakpoint_t breakpoint_t;
 
-breakpoint_t g_breakpoint;
+#define MAX_BREAKPOINTS 100
+breakpoint_t g_breakpoints[MAX_BREAKPOINTS];
 
 static trap_breakpoint_callback_t g_callback;
 
@@ -34,18 +35,28 @@ static void breakpoint_set(trap_inferior_t inferior, breakpoint_t *bp)
   ptrace_util_poke_text(pid, bp->aligned_address, modified_word);
 }
 
+breakpoint_t *breakpoint_alloc() {
+  // linear search for unused breakpoint
+  for (breakpoint_t *bp = g_breakpoints; bp < g_breakpoints[MAX_BREAKPOINTS]; bp++) {
+    if (bp->target_address == 0) {
+      // This is free, take it.
+      return bp;
+    }
+  }
+  assert(!"Failed to find a free breakpoint!");
+}
+
 trap_breakpoint_t trap_inferior_set_breakpoint(trap_inferior_t inferior,
                                                char *location)
 {
   const uintptr_t target_address = (uintptr_t)location;
   uintptr_t aligned_address = target_address & ~(0x7UL);
-  breakpoint_t *bp = &g_breakpoint;
+  breakpoint_t *bp = breakpoint_alloc();
 
   bp->original_breakpoint_word = ptrace_util_peek_text(inferior,
 						       aligned_address);
   bp->target_address = target_address;
   bp->aligned_address = aligned_address;

   breakpoint_set(inferior, bp);
 
   return bp;

Next, we need to rethink breakpoint_resolve, it’s used like this:

static trap_breakpoint_t breakpoint_resolve(trap_inferior_t inferior)
{
  return &g_breakpoint;
}

// ...

static void step_over_breakpoint(trap_inferior_t inferior)
{
  struct user_regs_struct regs;
  pid_t pid = inferior;

  trap_breakpoint_t bp = breakpoint_resolve(inferior);      // <===== Here ====
  breakpoint_remove(inferior, bp);
  breakpoint_trigger_callback(inferior, bp);

  ptrace_util_get_regs(pid, &regs);
  ptrace_util_set_instruction_pointer(pid, regs.rip - 1);

  ptrace_util_single_step(pid);
}

static void finish_breakpoint(trap_inferior_t inferior)
{
  breakpoint_t *bp = breakpoint_resolve(inferior);          // <====== Here ====
  breakpoint_set(inferior, bp);
  ptrace_util_continue(inferior);
}

enum inferior_state_t breakpoint_handle(trap_inferior_t inferior, enum inferior_state_t state)
{
  switch(state) {
    case INFERIOR_RUNNING:
      step_over_breakpoint(inferior);
      return INFERIOR_SINGLE_STEPPING;

    case INFERIOR_SINGLE_STEPPING:
      finish_breakpoint(inferior);
      return INFERIOR_RUNNING;

    default:
      abort();
  }
}

In step_over_breakpoint breakpoint_resolve is fine. That is, the inferior was running normally and stopped at a normal breakpoint and want to step over it. Inferior’s instruction pointer points to the end of the breakpoint instruction (0xCC) so one byte after the breakpoint target address.

But, in finish_breakpoint we have just finished single stepping and the inferior’s instruction pointer points to the start of the next instruction.

I’m going to remove the breakpoint_resolve call from breakpoint_finish. But, what do I replace it with? You might think that we could subtract something other than 1 from the instruction pointer to find the corresponding target_address. There are two things that make this impossible:

x86 has variable sized instructions. We could figure out the size of the instruction if we were at the start of the instruction by decoding it. But, we are past that instruction so we have no chance of decoding it.
The instruction that was just executed could have been some sort of branching instruction (e.g jump or call) so that by the end the instruction pointer has branched far away from the start of the original instruction.

So, basically the inferior’s current instruction pointer really tells us nothing about the address of the instruction that we just finished stepping over. I think this means that we have to save the breakpoint across the single step. There are two ways we can do this:

Stash it in a global. This is easy but I’m worried that as we develop new features in trap that this will be unsafe. I can’t quite imagine the scenarios that will lead to problems but it doesn’t feel right.
Restructure the breakpoint handling. I wonder if breakpoint_handle shouldn’t return to trap_inferior_continue instead it could call waitpid itself. Then, it can just keep the breakpoint it found initially.

This isn't a matter of making breakpoint_resolve smarter; it's a structural problem requiring us to maintain state across operations.

So I’m proposing a structural change that would look something like this:

enum inferior_state_t breakpoint_handle(trap_inferior_t inferior, enum inferior_state_t state)
{
  int status;

  // This is the critical fix - maintaining breakpoint state
  // across the single-step operation
  trap_breakpoint_t bp = breakpoint_resolve(inferior);

  assert(state == INFERIOR_RUNNING);
  step_over_breakpoint(inferior, bp);

  waitpid(inf->pid, &status, 0);
  if (WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP) {
    finish_breakpoint(inferior, bp);
  } else if (WIFEXITED(status)) {
    return;
  } else {
    fprintf(stderr, "Unexpected stop in trap_inferior_continue: 0x%x\n", status);
    abort();
  }
}

Ok, so to explain again: trap_inferior_continue will wake up when the breakpoint is triggered and call this new breakpoint_handle which will:

Look up (or resolve) the breakpoint.
Call step_over_breakpoint to remove the breakpoint and call PTRACE_SINGLESTEP.
Call waitpid to wait for the inferior to finish single stepping.
Call finish_breakpoint to restore the original breakpoint.

So this way breakpoint_handle doesn’t return back to trap_inferor_continue between single stepping and finishing. This means it can maintain state across the single step by simply keeping it on the stack. I think this is our solution.

Now we just have to fix breakpoint_finish to take the breakpoint, bp, as an argument (I’ll leave that as an exercise for the reader) and implement a new breakpoint_resolve to look the address up out of the g_breakpoints array.

static trap_breakpoint_t breakpoint_resolve(trap_inferior_t inferior)
{
  pid_t pid = inferior;
  struct user_regs_struct regs;
  uintptr_t target_address;

  ptrace_util_get_regs(pid, &regs);
  target_address = regs.rip - 1;  // Back up to the start of the breakpoint

  // linear search for target_address
  for (breakpoint_t *bp = g_breakpoints; bp < &g_breakpoints[MAX_BREAKPOINTS]; bp++) {
    if (bp->target_address == target_address) {
      return bp;
    }
  }
  assert(!"Failed to find existing breakpoint at address");
}

With a few more minor changes the tests pass!

The changes are to breakpoint_handle to:

Take over the responsibility of calling resolve_breakpoint. Pass bp to both step_over_breakpoint and finish_breakpoint.
Clean up some types
Make sure to return a value reflecting the current state in all successful branches.

enum inferior_state_t breakpoint_handle(trap_inferior_t inferior, enum inferior_state_t state)
{
  pid_t pid = inferior;

  // This is the critical fix - maintaining breakpoint state
  // across the single-step operation
  trap_breakpoint_t bp = breakpoint_resolve(inferior);
  int status;

  assert(state == INFERIOR_RUNNING);
  step_over_breakpoint(inferior, bp);
  waitpid(pid, &status, 0);
  if (WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP) {
    finish_breakpoint(inferior, bp);
    return INFERIOR_RUNNING;
  } else if (WIFEXITED(status)) {
    // This is a lie, but it should cause trap_inferior_continue to wait again.
    return INFERIOR_RUNNING;
  } else {
    fprintf(stderr, "Unexpected stop in trap_inferior_continue: 0x%x\n", status);
    abort();
  }
}

Next Steps

We have fixed our bug by making a structural change to breakpoint_handle that merges what was two calls into a single call that can maintain state about the breakpoint throughout the entire handling process. The old test that we originally wrote now passes.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

While I have put together a PR for this change we really should consider two options now. We can continue retracing our steps through Handling multiple breakpoints in Trap or we can take what we learned and fix the bug directly in the master branch.

Assuming the rest of the changes in the original Handling multiple breakpoints in Trap were good then I think porting the fix forward to the master branch makes the most sense. That’s not to say that the idea to retrace our steps was a bad one. There a number of advantages in resetting back to an old commit to debug. For me, I was able to look at the change fresh to better see what was going wrong. And from a test first perspective, resetting insures that we start from a passing test and can add one failing test at a time.

And while resetting has these advantages we should also keep in mind that once we leverage these advantages to solve the problem we shouldn’t let ourselves become overly attached to the new branch. If taking our new understanding and porting our fix forward yields quicker results then we should definitely do that. Though we must insure that we validate the fix properly in the master branch.

And I think that’s the case here because there are a lot of steps left in the roadmap we have based on the summary of the old post. We have done things like:

Removing Breakpoints
Deferred Removal
Dealing with setting a duplicate breakpoint.
Clean up

By porting forward we can keep all of these features and their tests. I’ll take care of porting the fix forward behind the scenes and end this post. This should leave trap in working state on my new development system. And as this post ends we should take away the following lessons:

Some bugs require a structural change to resolve especially when managing state throughout a longer process.
The technique of resetting and retracing our steps is a powerful one that can help us better focus on the problem in front on us. And once that problem is solved we should reevaluate the need to continue retracing vs porting the fix forward.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

The Official Relaunch of "A System Programming Blog"

Wed, 16 Jul 2025 00:00:00 +0000

Wow—exactly 10 years ago, I launched this blog. And while it only lived for a few short months, the original intent still resonates.

Today, I’m relaunching it with the same goals at heart:

To explore the Rust programming language
To demonstrate how to build low-level systems projects in C
To investigate emerging tools and technologies
To write long-form technical series
And to occasionally dive into smaller, focused topics

I’ve already written two new posts in which I get back up to speed on the debugger project and get the two projects building again.

And here’s what’s coming up:

I’ll finish work on the how to write a debugger series, likely with a more streamlined scope.
I’m launching a new series on Systems Thinking—these posts will appear between deeper technical dives.
I’ll also begin a new series on building a dynamic loader, using it to understand how executable loading works—and maybe even build a full binary interpreter.

Whether you’re new here or remember the original launch, thanks for being part of this reboot. This time, I’m going the distance.

Question and Answer Thinking

Wed, 16 Jul 2025 00:00:00 +0000

In this post I want to describe one form of Systems Thinking that I apply in my work and honestly in many aspects of my life. This is what I call Question and Answer thinking.

Whenever I start investigating an issue, solving a problem, or thinking about a new system to build I start by asking myself a question. I keep a notes document and I write this question down. Then, I take that question as a heading and start trying to answer it.

The first question is usually very broad, extremely broad. If I’m looking a bug in an OpenGL driver implementation in which the wrong image is rendered I simply ask: why did it draw this? If I’m looking at a high latency issue in a component of an AV system I might ask how does this component spend its time?

Are these questions helpful? These questions are obvious: of course we don’t know why the wrong image is drawn or how the time is spent. Or maybe we do know why in which case the question is even less helpful, right?

I find these questions and writing down the questions to be helpful in setting the context of the investigation, or design, or whatever work I’m trying to do. Writing down the questions sets an implicit goal of trying to write down an answer. When we arrive at an answer, writing it down is very helpful in that it serves to

Document something true, and apparently not obvious, about our system.
Helps to maintain context of what we know about the system
Directs further investigation

Ok, so we have a broad, obvious question written down which sets a goal to find an answer and document truth. Great, now what? Well, I usually start by asking a smaller question? Let’s continue with the example of looking at performance in an AV system.

Q: How does this component spend its time?
- Q: What tools do I have for investigating how a component spends its time?
  - A: I have a tracing tool to extract and visualize timing data.
  - I’ve run the tracing tool and generated mycomponent.trace.html.
- Q: What does the trace say?
  - A: The trace shows a lot of time spent in function foo?

So, this is an example refining questions and getting answers. I have questions and answers, and also maybe some other notes I took along the way explaining what I did or things that were unexpected. Now, I like to do this in a system that allows for folding. So I can fold closed questions. I’ve done this in org-mode and in Google docs. I also like to do a little curating of the answered questions by reordering the question and the answer. That is, when I have an answer I make the answer the headline and put the question under. If I do that I might end up with:

A: The component spends a lot of time in function foo.
- Q: How does this component spend its time?
- A: I have a tracing tool to extract and visualize timing data.
  - Q: What tools do I have for investigating how a component spends its time?
  - I’ve run the tracing tool and generated mycomponent.trace.html.
- A: The trace shows a lot of time spent in function foo?
  - Q: What does the trace say?

Now, depending on the tools you are using this curating can be a little tedious. I’ll admit that I found it tedious just now while writing the example. That said, in a real investigation you might spend quite a bit of time, say, reading and interpreting the trace so maybe this tedium is broken up by real work and overhead is low.

The advantage of curating the answers to the top is that when folded it reads like this:

A: The component spends a lot of time in function foo.
- A: I have a tracing tool to extract and visualize timing data.
- A: The trace shows a lot of time spent in function foo?

On one level, this is as I said before in that the answers document true and not so obvious facts about our system. But when we use this system of refinement, hide the questions, and just look at the answers we have not just a set of true statements but a true statement that answered our big broad question and a series of justifications of that fact. Further, we can open up any level to find notes on how we arrived at these conclusions.

This is very powerful.

Imagine you were working on a hard problem that’s going to take a while, and you get pulled off of it to work on some high priority bug. When you come back to the hard problem you can review a series of true statements about the problem that summarize what you know so far organized as a line of reasoning that you believe is directed toward the solution to your problem.

One thing to note, just because I’ve answered the initial broad obvious question doesn’t mean I’m done. In our performance example we concluded “The component spends a lot of time in function foo.” but so what? This doesn’t fix the problem or even really tell us how to fix the problem. But, it might help us figure out how to fix the problem if we ask more questions given our new knowledge. On one hand maybe we should have asked an even broader question first, on the other it doesn’t really matter. I have no objection to adding more broad questions at the top level. I can continue with any or all of:

How much time should we spend in the function foo?
Are we hitting a particularly bad case in foo?
How does the component spend its time in a well performing case?

Answering these new questions will give me a better understanding of the system and what went wrong in this particular issue.

Now, I’ve been doing this for… decades, I guess. And at this point I not only take notes this way but I think this way. This is how I approach problems: I ask a question and either answer it or break the question down into smaller questions until I know the answer or know how to find the answer. Finding the answer can mean

Read some code
Stop something in a debugger and inspect the state
Ask someone
Etc.

But if I know how to find the answer I do that. I write down what I found in the debugger or what my coworker told me in the notes and the set the answer. I continue this way until the problem is solved.

Now, this idea of “I ask a question and either answer it or break the question down into smaller questions until I know the answer” is very similar to the way I write software. We breakdown functionality using successive refinement making smaller and smaller functions until the function are easy to implement.

Similarly to writing this code, this strategy is good for large problems. It’s overkill for things that are very simple. Starting a doc, writing down questions, curating answers doesn’t make sense if you already know what is wrong. If you just wrote some new code and now it crashed and you can realize, “oops I passed the wrong argument here and it’s going to dereference a NULL pointer in this case” then just fix it. Don’t write these notes and don’t refine any questions. But, if you have a large problem to work on that might take a few days, or longer, then this system can be very powerful and this approach to systems thinking can really help you dig into problems, navigate systems, and build amazing things.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

Cleaning a rusty_trap.

Wed, 09 Jul 2025 00:00:00 +0000

In my last post I came back to my debugger project and got set up to build it and tried to get the tests to pass. In this post I’m doing the same with the rust version, rusty_trap.

I want to get in the habit of tracking where I’m starting these posts. So I’m at commit 6c22cba.

When I try to compile I get a few errors:

-*- mode: compilation; default-directory: "/plink:wsl:/home/jkain/projects/rusty_trap/" -*-
Compilation started at Sun Jul  6 08:53:57

/home/jkain/.cargo/bin/cargo b
warning: no edition set: defaulting to the 2015 edition while the latest is 2024
   Compiling nix v0.3.9 (https://github.com/joekain/nix-rust.git?branch=WaitStatus#4ae37a8b)
error[E0591]: can't transmute zero-sized type
   --> /home/jkain/.cargo/git/checkouts/nix-rust-9e641bdbdbe8194f/4ae37a8/src/sched.rs:168:20
	|
168 | ...   ffi::clone(mem::transmute(callback), ptr as *mut c_void, flags, &...
	|                  ^^^^^^^^^^^^^^
	|
	= note: source type: for<'a> extern "C" fn(*mut Box<(dyn FnMut() -> isize + 'a)>) -> i32 {callback}
	= note: target type: *const for<'a> extern "C" fn(*const Box<(dyn FnMut() -> isize + 'a)>) -> i32
	= help: cast with `as` to a pointer instead

For more information about this error, try `rustc --explain E0591`.
error: could not compile `nix` (lib) due to 1 previous error

Compilation exited abnormally with code 101 at Sun Jul  6 08:53:58

This is a problem building the nix crate which seems unexpected. I think I have some old version locked. Now that I think about it, everything must have some old version locked. But, looking at Cargo.toml I have:

[package]
name = "rusty_trap"
version = "0.1.0"
authors = ["Joseph Kain <joekain@gmail.com>"]

[dependencies]
libc     = "0.1.8"

[dependencies.nix]
git = "https://github.com/joekain/nix-rust.git"
version = "0.3.8"
branch = "WaitStatus"

[lib]
name = "rusty_trap"
path = "src/lib.rs"

[[bin]]
name = "twelve"
path = "tests/inferiors/twelve.rs"

[[bin]]
name = "loop"
path = "tests/inferiors/loop.rs"

So i have nix setup to track my repo with my WaitStatus branch, which is woefully out of date.

Now, I think there might have been a change I made that I never created a pull request for in proper nix. But, let’s see what happens if I just list nix as a dependency and pick of the officilal version. And I’ll do a cargo update to get new versions of all the dependencies.

@@ -4,12 +4,8 @@ version = "0.1.0"
 authors = ["Joseph Kain <joekain@gmail.com>"]
 
 [dependencies]
-libc     = "0.1.8"
-
-[dependencies.nix]
-git = "https://github.com/joekain/nix-rust.git"
-version = "0.3.8"
-branch = "WaitStatus"
+libc = "0.2.174"
+nix = "0.30.1"

Ok, this still doesn’t build but at least it’s complaining about problems in rusty_trap now.

-*- mode: compilation; default-directory: "/plink:wsl:/home/jkain/projects/rusty_trap/" -*-
Compilation started at Sun Jul  6 09:08:55

/home/jkain/.cargo/bin/cargo b
warning: no edition set: defaulting to the 2015 edition while the latest is 2024
 Downloading crates ...
  Downloaded cfg_aliases v0.2.1
  Downloaded nix v0.30.1
   Compiling cfg_aliases v0.2.1
   Compiling libc v0.2.174
   Compiling cfg-if v1.0.1
   Compiling bitflags v2.9.1
   Compiling nix v0.30.1
   Compiling rusty_trap v0.1.0 (/home/jkain/projects/rusty_trap)
error[E0432]: unresolved import `nix::unistd::Fork`
 --> src/lib.rs:5:18
  |
5 | use nix::unistd::Fork::*;
  |                  ^^^^ could not find `Fork` in `unistd`

error[E0432]: unresolved import `nix::sys::wait`
 --> src/lib.rs:9:15
  |
9 | use nix::sys::wait::*;
  |               ^^^^ could not find `wait` in `sys`

error[E0433]: failed to resolve: could not find `ptrace` in `sys`
  --> src/ptrace_util/mod.rs:3:15
   |
3  | use nix::sys::ptrace::ptrace::*;
   |               ^^^^^^ could not find `ptrace` in `sys`
   |
note: found an item that was configured out
  --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/sys/mod.rs:81:13
   |
81 |     pub mod ptrace;
   |             ^^^^^^
note: the item is gated behind the `ptrace` feature
  --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/sys/mod.rs:79:8
   |
79 |     #![feature = "ptrace"]
   |        ^^^^^^^

error[E0432]: unresolved import `nix::sys::ptrace`
 --> src/ptrace_util/mod.rs:2:15
  |
2 | use nix::sys::ptrace::*;
  |               ^^^^^^ could not find `ptrace` in `sys`

warning: unused import: `nix::unistd::*`
 --> src/lib.rs:4:5
  |
4 | use nix::unistd::*;
  |     ^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: unused import: `handle`
  --> src/lib.rs:22:18
   |
22 | use breakpoint::{handle, TrapBreakpoint};
   |                  ^^^^^^

warning: `extern` declarations without an explicit ABI are deprecated
  --> src/lib.rs:43:5
   |
43 |     extern {
   |     ^^^^^^ help: explicitly specify the "C" ABI: `extern "C"`
   |
   = note: `#[warn(missing_abi)]` on by default

error[E0599]: no variant or associated item named `Sys` found for enum `Errno` in the current scope
  --> src/lib.rs:82:24
   |
82 |             Err(Error::Sys(errno::EAGAIN)) => continue,
   |                        ^^^ variant or associated item not found in `Errno`

warning: variable does not need to be mutable
  --> src/breakpoint/mod.rs:35:34
   |
35 | pub fn handle<F>(inf: Inferior,  mut callback: &mut F) -> InferiorState
   |                                  ----^^^^^^^^
   |                                  |
   |                                  help: remove this `mut`
   |
   = note: `#[warn(unused_mut)]` on by default

Some errors have detailed explanations: E0432, E0433, E0599.
For more information about an error, try `rustc --explain E0432`.
warning: `rusty_trap` (lib) generated 4 warnings
error: could not compile `rusty_trap` (lib) due to 5 previous errors; 4 warnings emitted

Compilation exited abnormally with code 101 at Sun Jul  6 09:08:56

So it seems like either my additions to nix for wait, and ptrace aren’t present or they moved. There is something about fork as well so maybe things just moved. Let’s take a look at the nix docs.

Fork, looks like this should be in lowercase. No idea why it was capitalized before but let’s fix that. Fixed.

Also, this use line gives us some hints about waitpid as well:

use nix::{sys::wait::waitpid,unistd::{fork, ForkResult, write}};

The documentation also poits out

Available on crate feature process only.

This is new, I guess, and also relevant beause just using these doesn’t work:

warning: no edition set: defaulting to the 2015 edition while the latest is 2024
   Compiling rusty_trap v0.1.0 (/home/jkain/projects/rusty_trap)
error[E0432]: unresolved imports `nix::sys::wait`, `nix::unistd::fork`, `nix::unistd::ForkResult`, `nix::unistd::errno`
   --> src/lib.rs:4:16
	|
4   | ...sys::wait::waitpid,unistd::{fork, ForkResult, Error, errno}, sys::si...
	|         ^^^^                   ^^^^  ^^^^^^^^^^         ^^^^^
	|         |                      |     |                  |
	|         |                      |     |                  no `errno` in `unistd`
	|         |                      |     |                  help: a similar name exists in the module: `Errno`
	|         |                      |     no `ForkResult` in `unistd`
	|         |                      no `fork` in `unistd`
	|         could not find `wait` in `sys`
	|
note: found an item that was configured out
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/sys/mod.rs:181:13
	|
181 |     pub mod wait;
	|             ^^^^
note: the item is gated behind the `process` feature
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/sys/mod.rs:180:8
	|
180 |     #![feature = "process"]
	|        ^^^^^^^
note: found an item that was configured out
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/unistd.rs:278:15
	|
278 | pub unsafe fn fork() -> Result<ForkResult> {
	|               ^^^^
note: the item is gated behind the `process` feature
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/unistd.rs:159:4
	|
159 | #![feature = "process"]
	|    ^^^^^^^
note: found an item that was configured out
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/unistd.rs:209:10
	|
209 | pub enum ForkResult {
	|          ^^^^^^^^^^
note: the item is gated behind the `process` feature
   --> /home/jkain/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/nix-0.30.1/src/unistd.rs:159:4
	|
159 | #![feature = "process"]
	|    ^^^^^^^
	= help: consider importing this module instead:
			nix::errno

Oh, I need to add this to the Cargo file on the nix dependency. Looks like I need something similar for ptrace. This is new to me and seems pretty cool that I can pick and choose pieces of nix (and other crates I assume) to include.

modified   Cargo.toml
@@ -5,7 +5,7 @@ authors = ["Joseph Kain <joekain@gmail.com>"]
 
 [dependencies]
 libc = "0.2.174"
-nix = "0.30.1"
+nix = {version = "0.30.1", features = ["process", "ptrace"]}
 
 [lib]
 name = "rusty_trap"

Now, ptrace is interesting. It looks like someone had a similar idea to my idea to create separate functions for each of the different ptrace requests (PTRACE_CONT, PTRACE_PEEKTEXT, etc.) with proper type checking. That’s fantastic.

These are perfect and I’ll use them directly.

Well, in the short term I’ll implement ptrace_util on top of them which will

Let me review the types and see how they compare to the types I choose.
Leave the code that uses ptrace_util alone for now. I like to mimize changes until we get the tests running and passing again.

Hmm, PEEKTEXT and POKETEXT don’t seem to have coresponding functions in the new nix ptrace. However, PEEKDATA and POKEDATA do have functions (read, write) and Linux doesn’t actually separate text and data so I should be able to use these instead.

And ok, now it builds at least. No idea if it works but these seem pretty straightfoward so I hope they do.

modified   Cargo.toml
@@ -5,7 +5,7 @@ authors = ["Joseph Kain <joekain@gmail.com>"]
 
 [dependencies]
 libc = "0.2.174"
-nix = "0.30.1"
+nix = {version = "0.30.1", features = ["process", "ptrace"]}
 
 [lib]
 name = "rusty_trap"

modified   src/ptrace_util/mod.rs
@@ -1,8 +1,6 @@
 use libc::pid_t;
-use nix::sys::ptrace::*;
-use nix::sys::ptrace::ptrace::*;
-use std::ptr;
-use libc::c_void;
+use nix::sys::ptrace;
+use nix::unistd::Pid;
 
 use inferior::InferiorPointer;
 
@@ -39,44 +37,49 @@ pub mod user {
 }
 
 pub fn trace_me() -> () {
-    ptrace(PTRACE_TRACEME, 0, ptr::null_mut(), ptr::null_mut())
+    ptrace::traceme()
         .ok()
         .expect("Failed PTRACE_TRACEME");
 }
 
 pub fn get_instruction_pointer(pid: pid_t) -> InferiorPointer {
-    let raw = ptrace(PTRACE_PEEKUSER, pid, user::regs::RIP as * mut c_void, ptr::null_mut())
+    let raw = ptrace::read_user(Pid::from_raw(pid), user::regs::RIP as ptrace::AddressType)
         .ok()
         .expect("Failed PTRACE_PEEKUSER");
     InferiorPointer(raw as u64)
 }
 
 pub fn set_instruction_pointer(pid: pid_t, ip: InferiorPointer) -> () {
-    ptrace(PTRACE_POKEUSER, pid, user::regs::RIP as * mut c_void, ip.as_voidptr())
+    ptrace::write_user(Pid::from_raw(pid), user::regs::RIP as ptrace::AddressType, ip.as_i64())
         .ok()
         .expect("Failed PTRACE_POKEUSER");
 }
 
 pub fn cont(pid: pid_t) -> () {
-    ptrace(PTRACE_CONT, pid, ptr::null_mut(), ptr::null_mut())
+    ptrace::cont(Pid::from_raw(pid), None)
         .ok()
         .expect("Failed PTRACE_CONTINUE");
 }
 
 pub fn peek_text(pid: pid_t, address: InferiorPointer) -> i64 {
-    ptrace(PTRACE_PEEKTEXT, pid, address.as_voidptr(), ptr::null_mut())
-        .ok()
-        .expect("Failed PTRACE_PEEKTEXT")
+    // From ptrace(2) regarding PTRACE_PEEKTEXT and PTRACE_PEEKDATA
+    //   Linux does not have separate text and data address spaces,
+    //   so these two operations are currently equivalent.
+    // So use ptrace::read which is ptrace(PTRACE_PEEKDATA, ...)
+    // An alterantive would be to use libc::ptrace.
+    ptrace::read(Pid::from_raw(pid),  address.as_voidptr())
+	.ok()
+	.expect("Failed PTRACE_PEEKTEXT")
 }
 
 pub fn poke_text(pid: pid_t, address: InferiorPointer, value: i64) -> () {
-    ptrace(PTRACE_POKETEXT, pid, address.as_voidptr(), value as * mut c_void)
-        .ok()
-        .expect("Failed PTRACE_POKETEXT");
+    ptrace::write(Pid::from_raw(pid), address.as_voidptr(), value)
+	.ok()
+	.expect("Failed PTRACE_POKETEXT")
 }
 
 pub fn single_step(pid: pid_t) -> () {
-    ptrace(PTRACE_SINGLESTEP, pid, ptr::null_mut(), ptr::null_mut())
+    ptrace::step(Pid::from_raw(pid), None)
         .ok()
         .expect("Failed PTRACE_SINGLESTEP");
 }

Now the compilation errors go back to lib.rs, I fixed the problems with use earlier but now it’s looking at the code itself. On of the main issues I faced was that execve changed the types it expects. I had a lot of trouble getting this right and I think part of this is I have forgotten almost all the rust I knew and I might be doing this wrong. But these type conversions to pass to nix::unistd::execve are too much IMHO.

fn exec_inferior(filename: &Path, args: &[&str]) -> () {
    // let c_filename = &CStr::from_ptr(filename.to_str().unwrap().as_ptr() as *const i8);
    let cstring_filename = CString::new(filename.to_str()
					.expect("Failed to get string from filename"))
	.expect("Failed to create CString from filename");
    let cstr_filename = CStr::from_ptr(cstring_filename.as_ptr());

    disable_address_space_layout_randomization();
    ptrace_util::trace_me();
    execve::<CString, CString>(cstr_filename, &[], &[])
        .ok()
        .expect("Failed execve");
    unreachable!();
}

I kind worked through this by brute force / trial and error and am not proud of it. But, let’s move on.

On the bright side, libc now includes personality and I can use that.

@@ -37,35 +34,32 @@ static mut global_breakpoint : Breakpoint = Breakpoint {
 };
 static mut global_inferior : Inferior = Inferior { pid: 0, state: InferiorState::Stopped };
 
-mod ffi {
-    use libc::{c_int, c_long};
-
-    extern {
-        pub fn personality(persona: c_long) -> c_int;
-    }
-}
-
 fn disable_address_space_layout_randomization() -> () {
     unsafe {
-        let old = ffi::personality(0xffffffff);
-        ffi::personality((old | 0x0040000) as i64);
+	let old = libc::personality(0xffffffff);
+	libc::personality((old | libc::ADDR_NO_RANDOMIZE) as u64);
     }
 }

Happy to be rid of the hard coded 0x0040000.

The rest were some easier updates to the types. I do wonder if I should push nix::unistd::Pid all the way up and be consistent about using it.

modified   src/lib.rs
@@ -1,15 +1,12 @@
 extern crate nix;
 extern crate libc;
 
-use nix::unistd::*;
-use nix::unistd::Fork::*;
+use nix::{Error, sys::wait::waitpid,unistd::{execve, fork, ForkResult}, sys::signal};
 use libc::pid_t;
-use nix::Error;
-use nix::errno;
 use nix::sys::wait::*;
-use std::ffi::CString;
+use nix::unistd::Pid;
+use std::ffi::{CString, CStr};
 use std::path::Path;
-use nix::sys::signal;
 
 mod ptrace_util;
 
@@ -19,7 +16,7 @@ use inferior::*;
 mod breakpoint;
 
 pub use self::breakpoint::trap_inferior_set_breakpoint;
-use breakpoint::{handle, TrapBreakpoint};
+use breakpoint::{TrapBreakpoint};
 
 #[derive(Copy, Clone)]
 struct Breakpoint {
@@ -37,35 +34,32 @@ static mut global_breakpoint : Breakpoint = Breakpoint {
 };
 static mut global_inferior : Inferior = Inferior { pid: 0, state: InferiorState::Stopped };
 
-mod ffi {
-    use libc::{c_int, c_long};
-
-    extern {
-        pub fn personality(persona: c_long) -> c_int;
-    }
-}
-
 fn disable_address_space_layout_randomization() -> () {
     unsafe {
-        let old = ffi::personality(0xffffffff);
-        ffi::personality((old | 0x0040000) as i64);
+	let old = libc::personality(0xffffffff);
+	libc::personality((old | libc::ADDR_NO_RANDOMIZE) as u64);
     }
 }
 
 fn exec_inferior(filename: &Path, args: &[&str]) -> () {
-    let c_filename = &CString::new(filename.to_str().unwrap()).unwrap();
+    // let c_filename = &CStr::from_ptr(filename.to_str().unwrap().as_ptr() as *const i8);
+    let cstring_filename = CString::new(filename.to_str()
+					.expect("Failed to get string from filename"))
+	.expect("Failed to create CString from filename");
     disable_address_space_layout_randomization();
     ptrace_util::trace_me();
-    execve(c_filename, &[], &[])
+    let cstr_filename = unsafe { CStr::from_ptr(cstring_filename.as_ptr()) };
+    execve::<CString, CString>(cstr_filename, &[], &[])
         .ok()
         .expect("Failed execve");
     unreachable!();
 }
 
-fn attach_inferior(pid: pid_t) -> Result<Inferior, Error> {
-    match waitpid(pid, None) {
-        Ok(WaitStatus::Stopped(pid, signal::SIGTRAP)) =>
-            return Ok(Inferior {pid: pid, state: InferiorState::Running}),
+fn attach_inferior(raw_pid: pid_t) -> Result<Inferior, Error> {
+    let nix_pid = Pid::from_raw(raw_pid);
+    match waitpid(nix_pid, None) {
+        Ok(WaitStatus::Stopped(pid, signal::Signal::SIGTRAP)) =>
+            return Ok(Inferior {pid: pid.into(), state: InferiorState::Running}),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => return Err(e)
     }
@@ -73,25 +67,25 @@ fn attach_inferior(pid: pid_t) -> Result<Inferior, Error> {
 
 pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior, Error> {
     loop {
-        match fork() {
-            Ok(Child)                      => exec_inferior(filename, args),
-            Ok(Parent(pid))                => {
-                unsafe { global_inferior = attach_inferior(pid).ok().unwrap() };
-                return Ok(pid)
+        match unsafe { fork() } {
+            Ok(ForkResult::Child)                      => exec_inferior(filename, args),
+            Ok(ForkResult::Parent{child: pid})         => {
+                unsafe { global_inferior = attach_inferior(pid.into()).ok().unwrap() };
+                return Ok(pid.into())
             },
-            Err(Error::Sys(errno::EAGAIN)) => continue,
-            Err(e)                         => return Err(e)
+            Err(Error::EAGAIN) => continue,
+            Err(e)             => return Err(e)
         }
     }
 }
 
-pub fn trap_inferior_continue<F>(inferior: TrapInferior, callback: &mut F) -> i8
+pub fn trap_inferior_continue<F>(inferior: TrapInferior, callback: &mut F) -> i32
     where F: FnMut(TrapInferior, TrapBreakpoint) -> () {
 
     let mut inf = unsafe { global_inferior };
     ptrace_util::cont(inf.pid);
     loop {
-        inf.state = match waitpid(inf.pid, None) {
+        inf.state = match waitpid(Pid::from_raw(inf.pid), None) {
             Ok(WaitStatus::Exited(_pid, code)) => return code,
             Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) =>
                 breakpoint::handle(inf, callback),

Now, it compiles. But the tests fail. Before even running the tests I get some loader issues:

Inconsistency detected by ld.so: ../sysdeps/x86_64/dl-machine.h: 541: elf_machine_rela_relative: Assertion `ELFW(R_TYPE) (reloc->r_info) == R_X86_64_RELATIVE' failed!
Inconsistency detected by ld.so: ../sysdeps/x86_64/dl-machine.h: 541: elf_machine_rela_relative: Assertion `ELFW(R_TYPE) (reloc->r_info) == R_X86_64_RELATIVE' failed!

These are weird, unless I ptrace pokeed something that I shouldn’t have. And, I am suspicious that the breakpoint offsets would have moved, so let’s look into that first.

I used gdb to find the proper address of the breakpoints. And this elimited the inconsistencies detected by ld.so.

But the tests still don’t pass. Well, one of them doesn’t:

running 3 tests
test it_can_exec ... ok
test it_can_handle_a_breakpoint_more_than_once ... FAILED
test it_can_set_breakpoints ... ok

failures:

---- it_can_handle_a_breakpoint_more_than_once stdout ----

thread 'it_can_handle_a_breakpoint_more_than_once' panicked at /home/jkain/projects/rusty_trap/src/lib.rs:93:17:
Unexpected stop on signal SIGSEGV in trap_inferior_continue.  State: 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Ok, so inferior_continue found the inferior stopped on SIGSEGV. We could very well cause this by patching/restoring the breakpoint word incorrectly.

Is this the first time the breakpoint triggered?

Using printouts it seems that we’ve handled the breakpoint twice already. But stopping on the breakpoint calls breakpoint::handle once and the single stepping calls it a second time. Let’s confirm with better prints.

in breakpoint::handle inf.state = 0
in breakpoint::handle inf.state = 2

Right, so 0 is Running and 2 is SingleStepping.

So, in breakpoint::handle when single stepping we would attempt to set the breakpoint again and then continue.

Oh, this is the test that only has a breakpoint in main. Why did I think it was the other one?

Given that the test of a breakpoint multiple times works but the one with a single breakpoint occurance does not, it seems that either the address is wrong or something is wrong with trying to write this. Let me look at the main function disassembly in gdb:

I do see one bug and have fixed it but it isn’t the problem I’m looking for.

modified   src/breakpoint/mod.rs
@@ -27,7 +27,7 @@ fn step_over(inferior: TrapInferior, bp: Breakpoint) -> () {

 fn set(inferior: TrapInferior, bp: Breakpoint) -> () {
     let mut modified = bp.original_breakpoint_word;
-    modified &= !0xFFi64 << bp.shift;
+    modified &= !(0xFFi64 << bp.shift);
  
  modified |= 0xCCi64 << bp.shift;
     poke_text(inferior, bp.aligned_address, modified);

I think this was never an issue because the target and aligned address were always the same (and therefor shift was 0) becauze functions are usually aligned to 16 bytes. Either that or I don’t understand the operator precedence here. Eitehr way I think this is a good change but it doesn’t fix the test.

At this point I took a diversion to setup a cleaner development system with a newer version of Ubuntu. And then came back to working on this on the new system. Strangely, the test that was failing changed: The basic breakpoint test worked but the multiple breakpoint test stopped working.

One issue is that again compilers and systems have changed so the breakpoint addresses have moved. Also, the addresses now have namespaces so the symbol name is loop::foo and it gets C++-name-mangled. With this new name I’m able to find the address in gdb and plug it into the test.

After that, I wanted to focus only on the one test that is failing. I commented out the other tests (there must a better way to do this). Then it_can_handle_a_breakpoint_more_than_once passed!?!?

I asked, are these tests run concurrently?

Yes, concurrent use of rusty_trap isn’t going to work yet.

I can use cargo test -- --test-threads=1 to run them sequentially.

Now this passes! Well, I learned something about cargo testing and we have new code smell to encourage us to make rusty_trap thread safe. The biggest road block here is the global breakpoint.

Also, from the same documentation I see that there is a better way to run a single test: cargo test it_can_handle_a_breakpoint_more_than_once

Here is the pull request for all of the changes we made for this post.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

There are a number of to do items we can carry forward with us.

TODO:

Go over the warnings
Consider nix::unistd::Pid rather than pid_t.
Look at ptrace::AddressType and consider replacing InferiorPointer.
Get rid of the global breakpoint and make the system thread safe.
Handle multiple breakpoints.

We should be able to tackle some of these in the next post. I hope to see you there.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

Trap - Back Up and Running

Tue, 08 Jul 2025 00:00:00 +0000

It’s been a long time and I’m finally coming back to this project and need to get things building again. I tried compiling the project and ran into the following errors:

-- Configuring done
-- Generating done
-- Build files have been written to: /home/jkain/projects/trap/_out
Scanning dependencies of target trap
[  4%] Building C object src/CMakeFiles/trap.dir/inferior_load.c.o
/home/jkain/projects/trap/src/inferior_load.c: In function ‘attach_to_inferior’:
/home/jkain/projects/trap/src/inferior_load.c:24:49: error: ‘SIGTRAP’ undeclared (first use in this function)
   24 |   if (WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP) {
  |                                                 ^~~~~~~
/home/jkain/projects/trap/src/inferior_load.c:24:49: note: each undeclared identifier is reported only once for each function it appears in
/home/jkain/projects/trap/src/inferior_load.c: In function ‘trap_inferior_continue’:
/home/jkain/projects/trap/src/inferior_load.c:71:51: error: ‘SIGTRAP’ undeclared (first use in this function)
   71 |     if (WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP) {
  |                                                   ^~~~~~~
make[2]: *** [src/CMakeFiles/trap.dir/build.make:63: src/CMakeFiles/trap.dir/inferior_load.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:149: src/CMakeFiles/trap.dir/all] Error 2
make: *** [Makefile:95: all] Error 2

SIGTRAP isn’t defined? I guess I need to include signal.h but I don’t understand why I didn’t need this before.

modified   src/inferior_load.c
@@ -5,6 +5,7 @@
 #include <sys/wait.h>
 #include <assert.h>
 #include <errno.h>
+#include <signal.h>
 #include <stdio.h>
 #include <stdlib.h>

Now it builds, great! But the tests fail:

Test project /home/jkain/projects/trap/_out
	Start 1: test_exec
1/7 Test #1: test_exec ............................   Passed    0.00 sec
	Start 2: test_breakpoint
2/7 Test #2: test_breakpoint ......................Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error
Hello World!

	Start 3: test_multi_breakpoint
3/7 Test #3: test_multi_breakpoint ................Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error

	Start 4: test_multiple_breakpoints
4/7 Test #4: test_multiple_breakpoints ............Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error

	Start 5: test_delete_breakpoint
5/7 Test #5: test_delete_breakpoint ...............Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error

	Start 6: test_delete_breakpoint_in_callback
6/7 Test #6: test_delete_breakpoint_in_callback ...Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error

	Start 7: test_duplicate_breakpoint
7/7 Test #7: test_duplicate_breakpoint ............Child aborted***Exception:   0.01 sec
PTRACE_POKETEXT: : Input/output error


14% tests passed, 6 tests failed out of 7

Total Test time (real) =   0.06 sec

The following tests FAILED:
	  2 - test_breakpoint (Child aborted)
	  3 - test_multi_breakpoint (Child aborted)
	  4 - test_multiple_breakpoints (Child aborted)
	  5 - test_delete_breakpoint (Child aborted)
	  6 - test_delete_breakpoint_in_callback (Child aborted)
	  7 - test_duplicate_breakpoint (Child aborted)
Errors while running CTest

Oh, this is because the address for the breakpoints has changed because it’s been 10 years and I’m using a different system, compiler, etc. now. This is a great reminder that we should implement symbol look up so that the breakpoints aren’t just hard coded addresses, but let’s get the basics working again before we take that on.

Fixing the offsets isn’t enough to fix this. There is probably address space randomization like we saw with Rust. I’ve implemented personality calls to disable address randomization.

@@ -12,6 +14,11 @@ static inferior_t g_inferior;
 
 static void setup_inferior(const char *path, char *const argv[])
 {
+  unsigned long old = personality(0xFFFFFFFF);
+  if (personality(old | ADDR_NO_RANDOMIZE) < 0) {
+    perror("Failed to set personality:");
+  }
+
   ptrace_util_traceme();
   execv(path, argv);
 }

But this still doesn’t work. So, what is the address of main?

From gdb main = 0x555555555169. And it is always this value, it’s not random. I am running on WSL, maybe that’s weird. Also does this binary have a start address set?

Yes, it does. The start address is 0x0000000000001080.

Which coresponds with the normal offset of .text

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
 15 .text         00000195  0000000000001080  0000000000001080  00001080  2**4
				  CONTENTS, ALLOC, LOAD, READONLY, CODE

Ok, so what’s going on? Maybe another look at gdb can help:

$ gdb inferiors/hello
(No debugging symbols found in inferiors/hello)
(gdb) b main
Breakpoint 1 at 0x1169
(gdb)

So gdb gets the basic address (note this is different than the 0x1149 we saw before because I added a printf call and string.).

Hang on, let’s run under gdb:

(gdb) r
Starting program: /home/jkain/projects/trap/_out/test/inferiors/hello

Breakpoint 1, 0x0000555555555169 in main ()
(gdb)

Ok, so object is being loaded into a different address as it runs. And at the start gdb doesn’t know where it will be loaded (just like trap).

For now, I can just hard code this address, I guess. I’ve removed the printout from the inferior and use 0x555555555149 as the breakpoint address.

Now that I think about it, I saw the same thing in Rust when I last worked on it. This makes sense in that the binary wouldn’t necessarily get loaded at address 0. If I understand correctly, there are non-relocatable types of binaries and that must have been what I was building before and the newer tool chain allows for relocation. But, that’s just speculation on my part.

Even with these new addresses the I see an assertion failure in breakpoint_resolve. I added a printout in breakpoint_resolve to print the instruction pointer. I see two calls:

breakpoint_resolve: ip = 0x555555555149
breakpoint_resolve: ip = 0x55555555514c
test_breakpoint: /home/jkain/projects/trap/src/breakpoint.c:102: breakpoint_resolve: Assertion `result' failed.

The first call to breakpoint_resolve must be when the breakpoint in main is reached. And the second call is 3 bytes later. I think this is trying to resolve the stop in execution that happens when we are in state INFERIOR_SINGLE_STEPPING to step over the breakpoint. And in this case the instruction pointer will have moved past the byte where we inserted the breakpoint – past the entire original instruction. So breakpoint_resolve should fail as this address isn’t one of our breakpoints. That is, breakpoint_resolve assumes we executed a one byte instruction and subtracts 1 from the instruction pointer to find the target_address.

This is not correct and wasn’t correct when I originally wrote it. I don’t know how this worked before.

Is this something I never wrote about (and presumably didn’t test properly)?

It looks like this was added in the multiple-breakpoints post.

I think I need to back up to the previous post and, make these breakpoint address tweaks, and then retrace my steps through multiple-breakpoints. I also should get in the habit of announcing which commit I’m starting with in each post and have a PR at the end of each post.

I’ll take this work on in a separate post. For now, I’ve pushed the fixes as a PR and we should get started on getting the Rust version working again.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

But, before, I go I have one question for you. At some point in the future I plan to port trap and rusty_trap to ARM. Do you have a favorite ARM board? I have a few: an old Raspberry Pi 2, an odroid XU4, and an Orange Pi Zero 3. I’m thinking about getting an Orange Pi 5. Let me know your thoughts in the comments.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

Refactor fail

Fri, 25 Sep 2015 00:00:00 +0000

In the last post we implemented a state machine to manage breakpoints in rusty_trap. We also added support for hitting a single breakpoint multiple times. Now, I want to start adding support for multiple distinct breakpoints.

One way to approach adding a new feature is to refactor to make the code easily accept the new feature. That’s what I set out to do today, but it didn’t go the way I planned.

Multiple Breakpoints

I still start with writing a test for my goal - to support multiple breakpoints. This will help me identify the refactorings I need:

#[test]
fn it_can_handle_multiple_breakpoints () {
    let mut foo_count:  i32 = 0;
    let mut main_count: i32 = 0;

    let inferior = rusty_trap::trap_inferior_exec(Path::new("./target/debug/loop"), &[])
        .unwrap();
    let bp_foo  = rusty_trap::trap_inferior_set_breakpoint(inferior, 0x5555555587b0);
    let bp_main = rusty_trap::trap_inferior_set_breakpoint(inferior, 0x555555558bd0);
    rusty_trap::trap_inferior_continue(inferior, &mut |passed_inferior, bp_passed| {
        assert_eq!(passed_inferior, inferior);
        if bp_passed == bp_foo {
            foo_count  += 1
        } else if bp_passed == bp_main {
             main_count += 1
        } else {
            panic!("Unknown breakpoint {}", bp_passed)
        }
    });

    assert_eq!(foo_count, 5);
    assert_eq!(main_count, 5)
}

and of course this test fails which is expected. Though the error isn’t exactly what I expected:

---- it_can_handle_multiple_breakpoints stdout ----
	thread 'it_can_handle_multiple_breakpoints' panicked at 'assertion failed: `(left == right)` (left: `2`, right: `5`)', tests/lib.rs:63

So it looks like we only detected 2 two hits to bp_foo. I would have expected to get all 5 an maybe more (due to misattribution of bp_main).

I also see this

test it_can_handle_multiple_breakpoints ... thread '<main>' panicked at 'assertion failed: c.borrow().is_none()', ../src/libstd/sys/common/thread_info.rs:54
FAILED

I’m not sure exactly what this is about or when it happened. The test must have gotten at least two assert_eq!(foo_count, 5);

Implementation

As I mentioned, the test should point me in the direction I need to refactor. So, I need to consider: hat do we need to do in order to implement this feature?

Looking over breakpoint/mod.rs I think we need to

Remove global_breakpoint in favor of a vector of breakpoints
Update breakpoint::handle to lookup the correct breakpoint
Fix trap_inferior_set_breakpoint to add to the vector of breakpoints

Also, instead of having a global vector of breakpoints. I’ll keep the vector in the Inferior. This will help when we need to support multiple inferiors. One problem is that this will create a circular dependency between inferior and breakpoint modules.

I need some private structure for breakpoint module that fits into Inferior. How do I do this in Rust? Maybe I can use a generic that lets me store/fetch it in a Vector. A little googling turns up that I can use the Any trait.

If I want to put a Vector inside Inferior then I will need to remove the Copy trait from Inferior. I don’t want to to be copying the Vectors. Given this I should remove global_inferior.

Remove global_inferior

I’ll do this by dynamically allocating an inferior and returning it to the application. This eliminates the need for Copy trait on Inferior.

I start by commenting out global_inferior and letting the compiler tell me what to fix:

Compiling rusty_trap v0.1.0 (file:///home/joseph/src/rust/rusty_trap)
src/lib.rs:79:26: 79:41 error: unresolved name `global_inferior` [E0425]
src/lib.rs:79                 unsafe { global_inferior = attach_inferior(pid).ok().unwrap() };
                                    ^~~~~~~~~~~~~~~
src/lib.rs:79:26: 79:41 help: run `rustc --explain E0425` to see a detailed explanation
src/lib.rs:91:28: 91:43 error: unresolved name `global_inferior` [E0425]
src/lib.rs:91     let mut inf = unsafe { global_inferior };
                                      ^~~~~~~~~~~~~~~
src/lib.rs:91:28: 91:43 help: run `rustc --explain E0425` to see a detailed explanation
src/lib.rs:105:18: 105:33 error: unresolved name `global_inferior` [E0425]
src/lib.rs:105         unsafe { global_inferior = inf };
                             ^~~~~~~~~~~~~~~
src/lib.rs:105:18: 105:33 help: run `rustc --explain E0425` to see a detailed explanation
error: aborting due to 3 previous errors
Could not compile `rusty_trap`.

Changes to inferior/mod.rs:

diff --git a/src/inferior/mod.rs b/src/inferior/mod.rs
index 17eaa77..e5eaa55 100644
--- a/src/inferior/mod.rs
+++ b/src/inferior/mod.rs
@@ -1,6 +1,9 @@
 use libc::pid_t;
 use libc::c_void;
 use std::ops::{Add, Sub};
+use std::any::Any;
+use std::vec::Vec;

 #[derive(Copy, Clone)]
 pub enum InferiorState {
@@ -9,13 +12,14 @@ pub enum InferiorState {
     SingleStepping
 }

-#[derive(Copy, Clone)]
+//#[derive(Copy, Clone)]
 pub struct Inferior {
     pub pid: pid_t,
-    pub state: InferiorState
+    pub state: InferiorState,
+    pub privates: Vec<Box<Any>>
 }

-pub type TrapInferior = pid_t;
+pub type TrapInferior = Box<Inferior>;

 #[derive(Copy, Clone)]
 pub struct InferiorPointer(pub u64);

Right now, the only important changes is the change to TrapInferior. The changes to Inferior are still work in progress.

Changes to lib.rs:

diff --git a/src/lib.rs b/src/lib.rs
index ac9e444..70ded88 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -29,13 +29,13 @@ struct Breakpoint {
     original_breakpoint_word : i64
 }

-static mut global_breakpoint : Breakpoint = Breakpoint {
-    shift: 0,
-    target_address: InferiorPointer(0),
-    aligned_address: InferiorPointer(0),
-    original_breakpoint_word: 0
-};
-static mut global_inferior : Inferior = Inferior { pid: 0, state: InferiorState::Stopped };
+// static mut global_breakpoint : Breakpoint = Breakpoint {
+//     shift: 0,
+//     target_address: InferiorPointer(0),
+//     aligned_address: InferiorPointer(0),
+//     original_breakpoint_word: 0
+// };
+//static mut global_inferior : Inferior = Inferior { pid: 0, state: InferiorState::Stopped, privates: vec![] };

 mod ffi {
     use libc::{c_int, c_long};
@@ -62,10 +62,10 @@ fn exec_inferior(filename: &Path, args: &[&str]) -> () {
     unreachable!();
 }

-fn attach_inferior(pid: pid_t) -> Result<Inferior, Error> {
+fn attach_inferior(pid: pid_t) -> Result<Box<Inferior>, Error> {
     match waitpid(pid, None) {
         Ok(WaitStatus::Stopped(pid, signal::SIGTRAP)) =>
-            return Ok(Inferior {pid: pid, state: InferiorState::Running}),
+            return Ok(Box::new(Inferior {pid: pid, state: InferiorState::Running, privates: vec![]})),
         Ok(_) => panic!("Unexpected stop in attach_inferior"),
         Err(e) => return Err(e)
     }
@@ -76,8 +76,8 @@ pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior
         match fork() {
             Ok(Child)                      => exec_inferior(filename, args),
             Ok(Parent(pid))                => {
-                unsafe { global_inferior = attach_inferior(pid).ok().unwrap() };
-                return Ok(pid)
+                let inf = attach_inferior(pid).ok().unwrap();
+                return Ok(inf)
             },
             Err(Error::Sys(errno::EAGAIN)) => continue,
             Err(e)                         => return Err(e)
@@ -86,22 +86,20 @@ pub fn trap_inferior_exec(filename: &Path, args: &[&str]) -> Result<TrapInferior
 }

 pub fn trap_inferior_continue<F>(inferior: TrapInferior, callback: &mut F) -> i8
-    where F: FnMut(TrapInferior, TrapBreakpoint) -> () {
+    where F: FnMut(&TrapInferior, TrapBreakpoint) -> () {

-    let mut inf = unsafe { global_inferior };
+    let mut inf = inferior;
     ptrace_util::cont(inf.pid);
     loop {
         inf.state = match waitpid(inf.pid, None) {
             Ok(WaitStatus::Exited(_pid, code)) => return code,
             Ok(WaitStatus::Stopped(_pid, signal::SIGTRAP)) =>
-                breakpoint::handle(inf, callback),
+                breakpoint::handle(&inf, callback),
             Ok(WaitStatus::Stopped(_pid, signal)) => {
               panic!("Unexpected stop on signal {} in trap_inferior_continue.  State: {}", signal, inf.state as i32)
           },
           Ok(_) => panic!("Unexpected stop in trap_inferior_continue"),
           Err(_) => panic!("Unhandled error in trap_inferior_continue")
       };
-
-        unsafe { global_inferior = inf };
   }
}

Here I’ve commened out global_inferior and allocated an Inferior dynamically in attach_inferior using Box. I’ve also started passing around references to the Box<Inferior> rather than moving the pointer.

Changes to breakpoint/mod.rs:

diff --git a/src/breakpoint/mod.rs b/src/breakpoint/mod.rs
index a069427..1d4aa1d 100644
--- a/src/breakpoint/mod.rs
+++ b/src/breakpoint/mod.rs
@@ -19,33 +19,33 @@ static mut global_breakpoint : Breakpoint = Breakpoint {
 };


-fn step_over(inferior: TrapInferior, bp: Breakpoint) -> () {
-    poke_text(inferior, bp.aligned_address, bp.original_breakpoint_word);
-    set_instruction_pointer(inferior, bp.target_address);
-    single_step(inferior);
+fn step_over(inferior: &TrapInferior, bp: Breakpoint) -> () {
+    let pid = inferior.pid;
+    poke_text(pid, bp.aligned_address, bp.original_breakpoint_word);
+    set_instruction_pointer(pid, bp.target_address);
+    single_step(pid);
 }

-fn set(inferior: TrapInferior, bp: Breakpoint) -> () {
+fn set(inferior: &TrapInferior, bp: Breakpoint) -> () {
     let mut modified = bp.original_breakpoint_word;
     modified &= !0xFFi64 << bp.shift;
     modified |= 0xCCi64 << bp.shift;
-    poke_text(inferior, bp.aligned_address, modified);
+    poke_text(inferior.pid, bp.aligned_address, modified);
 }

-pub fn handle<F>(inf: Inferior,  mut callback: &mut F) -> InferiorState
-    where F: FnMut(TrapInferior, TrapBreakpoint) -> () {
-    let inferior = inf.pid;
+pub fn handle<F>(inf: &TrapInferior,  mut callback: &mut F) -> InferiorState
+    where F: FnMut(&TrapInferior, TrapBreakpoint) -> () {

     let bp = unsafe { global_breakpoint };
     match inf.state {
         InferiorState::Running => {
-            callback(inferior, 0);
-            step_over(inferior, bp);
+            callback(&inf, 0);
+            step_over(&inf, bp);
             InferiorState::SingleStepping
         },
         InferiorState::SingleStepping => {
-            set(inferior, bp);
-            cont(inferior);
+            set(&inf, bp);
+            cont(inf.pid);
             InferiorState::Running
         },
         _ => panic!("Unsupported breakpoint encountered during supported inferior state")
@@ -58,10 +58,10 @@ pub fn trap_inferior_set_breakpoint(inferior: TrapInferior, location: u64) -> Tr
         shift : (location - aligned_address) * 8,
         aligned_address: InferiorPointer(aligned_address),
         target_address: InferiorPointer(location),
-        original_breakpoint_word: peek_text(inferior, InferiorPointer(aligned_address))
+        original_breakpoint_word: peek_text(inferior.pid, InferiorPointer(aligned_address))
     };

-    set(inferior, bp);
+    set(&inferior, bp);

     unsafe {
         global_breakpoint = bp;

These changes are mostly of one of two forms:

Taking the pid field from the Inferior rather than assuming InferiorTrap is a pid.
Passing references to the Box<Inferior> around instead of moving. Previously these were copies. Passing references fixes up ownership now that we don’t copy.

These changes all seem well and good until I get to the tests. I expected to have to fix up a few things (in fact the API changed in thatthe callback now takes a reference). But I didn’t understand the real problem until I saw the error:

$ cargo test
   Compiling rusty_trap v0.1.0 (file:///home/joseph/src/rust/rusty_trap)
<std macros>:5:8: 5:18 error: binary operation `==` cannot be applied to type `&Box<rusty_trap::inferior::Inferior>` [E0369]
<std macros>:5 if ! ( * left_val == * right_val ) {
                      ^~~~~~~~~~
<std macros>:1:1: 9:39 note: in expansion of assert_eq!
tests/lib.rs:19:9: 19:47 note: expansion site
note: in expansion of closure expansion
tests/lib.rs:18:55: 22:6 note: expansion site

So, it seems that these references are not comparable. Which I guess makes some sense. I actually, shouldn’t have more than one copy of the pointer or reference to the pointer in the first place. That must be violating the borrow rules.

Enjoying the post? Sign up for the newsletter or support the work on Buy Me a Coffee .

Next steps

I can’t really proceed with this refactoring. I’m going to need to go back to where I started and try something different. I’m thinking I will go back to returning the pid, or some opaque handle, to the application and then using that to lookup the Inferior data structure.

I’ll try again next week.

This post is part of my Writing a Debugger series.
View all posts and upcoming topics →

A System Programming Blog

Demangling the Details: Symbol Resolution in Rusty Trap

Exploring ELF Files and Symbol Tables

Using the object Crate for Symbol Resolution

Tests for Symbol Resolution

Passing symbol names through rusty_trap

Lifetimes and Self-Referential Structs

Encapsulating Validated File Names and File Data

Implementing Symbol Resolution

Finding the Base Address

Lessons Learned

Vibe Coding an API Refactor

Me: Initial Ask

Claude: On Returning the Inferior from trap_inferior_continue

Reasons to Not Return the Inferior

Reasons to Return the Inferior

Recommended Approach

An aside: My Thoughts returning the inferior from trap_inferior_continue

Me: Let’s move on

Claude: Implementing Ownership-Based Debugger API

Updated trap_inferior_set_breakpoint Function

Updated trap_inferior_continue Function

Updating the breakpoint::handle Function

Updated Test Example

Multiple Breakpoints Test

Aside: Review

Takeaways

Multiple Breakpoints in Rust: Ownership-Driven Debugger Design

Approaching the Problem With a Clear Mind

Aside on API design?

Conclusion: From Global State to Ownership

What’s Next?

How I read new code

Ring Theory

Reading Code

Top down / Bottom up

Wrapping Up

Handling multiple breakpoints in Trap Redux

AI Generated Summary

Initial Goals

Implementing Multiple Breakpoints Support

Resolving the Right Breakpoint

Handling Duplicate Breakpoints

Removing Breakpoints

Key Bug: Duplicate Breakpoint Reading

Deferred Removal Implementation

Refactoring and Cleanup

Final Issues

My Summary of the Summary

Retracing Our Steps

Implementing Multiple Breakpoints with Working Resolution

Next Steps

The Official Relaunch of "A System Programming Blog"

Question and Answer Thinking

Cleaning a rusty_trap.

Trap - Back Up and Running

Refactor fail

Multiple Breakpoints

Implementation

Remove global_inferior

Next steps

Claude: On Returning the Inferior from `trap_inferior_continue`

An aside: My Thoughts returning the inferior from `trap_inferior_continue`

Updated `trap_inferior_set_breakpoint` Function

Updated `trap_inferior_continue` Function

Updating the `breakpoint::handle` Function