--- date: 2012-10-27T00:00:00-05:00 title: "GDB and SystemTap Probes -- part 2" tags: [en_us, english, gdb, systemtap, howto, fedora-planet, free-software] --- I tell you this: it is depressing when you realize that you spent more time struggling with blog engines than writing posts on your blog! It's been a long time since I wrote the [first post]({filename}/2012-03-29-gdb-and-systemtap-probes-part-1.md) about this subject, and since then the patches have been accepted upstream, and [GDB 7.5](http://www.gnu.org/software/gdb/download/ANNOUNCEMENT) now has official support for userspace SystemTap probes :-). Yay! Well, but enough of cheap talk, let's get to the business! Errata for my last post ----------------------- [Frank Ch. Eigler](http://web.elastic.org/~fche/blog2/), one of SystemTap's maintainers, kindly mentioned something that I should say about SystemTap userspace probes. Basically, it should be clear that `SDT` probes are not the only kind of userspace probing one can do with SystemTap. There is yet another kind of probe (maybe even more powerful, depending on the goals): **DWARF-based function/statement probes**. SystemTap supports this kind of probing mechanism for quite a while now. It is not the goal of this post to explain it in detail, but you might want to give it a try by compiling your binary with debuginfo support (use the `-g` flag on `GCC`), and do something like: ```console $ stap -e 'probe process("/bin/foo").function("name") { log($$parms) }' -c /bin/foo $ stap -e 'probe process("/bin/foo").statement("*@file.c:443") { log($$vars) }' -c /bin/foo ``` And that's it. You can read SystemTap's documentation, or [this](http://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps) guide to learn how to add userspace probes. Using GDB with SystemTap SDT Probes ----------------------------------- Well, now let's get to the interesting part. It is time to make `GDB` work with the `SDT` probe that we have put in our example code. Let's remember it: ```c #include int main (int argc, char *argv[]) { int a = 10; STAP_PROBE1 (test_program, my_probe, a); return 0; } ``` It is a very simple example, and we will have to extend it later in order to show more features. But for now, it will do. The first thing to do is to open `GDB` (with SystemTap support, of course!), and check to see if it can actually see probe inserted in our example. ```console $ gdb ./test_program GNU gdb (GDB) 7.5.50.20121014-cvs Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later ... (gdb) info probes Provider Name Where Semaphore Object test_program my_probe 0x00000000004004ae /home/sergio/work/src/git/build/gdb/test_program ``` Wow, it actually works! :-) If you have seen something like the above, it means your `GDB` is correctly recognizing `SDT` probes. If you see an error, or if your `GDB` doesn't have the `info probes` command, then you'd better make sure you have a recent version of `GDB` otherwise you won't be able to use the `SDT` support. ### Putting breakpoints in the code Anyway, now it is time to start using this support. The first thing I want to show you is how to put a breakpoint in a probe. ```console (gdb) break -probe-stap my_probe Breakpoint 1 at 0x4004ae ``` That's all! We have chosen to extend the `break` command in order to support the new `-probe-stap` parameter. If you're wondering *... why the -probe prefix?*, it is because I was asked to implement a complete abstraction layer inside `GDB` in order to allow more types of probes to be added in the future. So, for example, if someone implements support for an hypothetical type of probe called `xyz`, you would have `break -probe-xyz`. It took me a little more time to implement this layer, but it is worth the effort. Anyway, as you have see above, `GDB` recognize the probe's name and correctly put a breakpoint in it. You can also confirm that it has done the right thing by matching the address reported by `info probes` with the one reported by `break`: they should be the same. Ok, so now, with our `breakpoint` in place, let's run the program and see what happens. ```console (gdb) run Starting program: /home/sergio/work/src/git/build/gdb/test_program Breakpoint 1, main (argc=1, argv=0x7fffffffdf68) at /tmp/example-stap.c:8 8 STAP_PROBE1 (test_program, my_probe, a); ``` As you can see, `GDB` stopped at the exact location of the probe. Therefore, you are now able to put marks (i.e., probes) in your source code which are **location-independent**. It means that it doesn't really matter where in the source code your probe is, and it also doesn't matter if you change the code around it, changing the line numbers, or even moving it to another file. `GDB` will always find your probe, and always stop at the right location. Neat! ### Examining probes' arguments But wait, there's more! Remember when I told you that you could also inspect the probe's arguments? Yes, let's do it now! Just remember that, in `SDT`'s parlance, the current probe's argument is `a`. So let's print its value. ```console (gdb) p $_probe_arg0 $1 = 10 (gdb) p a $2 = 10 ``` *"Hey, captain, it seems the boat really floats!"* Check the source code above, and convince yourself that `a`'s value is `10` :-). As you might have seen, I have used a fairly strange way of printing it. It is because the probe's arguments are available inside `GDB` by means of **convenience variables**. You can see a list of them [here](http://sourceware.org/gdb/current/onlinedocs/gdb/Convenience-Vars.html#Convenience-Vars). Since `SDT` probes can have up to 12 arguments (i.e., you can use `STAP_PROBE1` ... `STAP_PROBE12`), we have created inside `GDB` 12 convenience variables, named `$_probe_arg0` until `$_probe_arg11`. I know, it is not an easy name to remember, and even the relation between `SDT` naming and `GDB` naming is not direct (i.e., you have to subtract 1 from the `SDT` probe number). If you are not satisfied with this, please open a bug in our [bugzilla](http://sourceware.org/bugzilla/) and I promise we will discuss other options. I would like to emphasize something here: just as you don't need debuginfo support for dealing with probes inside `GDB`, you also don't need debuginfo support for dealing with their arguments as well. It means that you can actually compile your code without debuginfo support, but still have access to some important variables/expressions when debugging it. Depending on how `GCC` optimizes your code, you may experience some difficulties with argument printing, but so far I haven't heard of anything like that. More to come ------------ Ok, now we have covered more things about the `SDT` probe support inside `GDB`, and I hope you understood all the concepts. It is not hard to get things going with this, specially because you don't need extra libraries to make it work. In the next post, I intend to finish this series by explaining how to use `tracepoints` with `SDT` probes. Also, as I said in the previous post of this series, maybe I will talk a little bit about how the `SDT` probes are organized within the binary. See you soon!