blog/content/posts/gdb-and-systemtap-probes-part-2.md

---
date: 2012-10-27T00:00:00-05:00
title: "GDB and SystemTap Probes -- part 2"
tags: [en_us, english, gdb, systemtap, howto, fedora-planet, free-software]
---

I tell you this: it is depressing when you realize that you spent more
time struggling with blog engines than writing posts on your blog!

It's been a long time since I wrote
the
[first post]({filename}/2012-03-29-gdb-and-systemtap-probes-part-1.md) about
this subject, and since then the patches have been accepted upstream,
and [GDB 7.5](http://www.gnu.org/software/gdb/download/ANNOUNCEMENT)
now has official support for userspace SystemTap probes :-). Yay!

Well, but enough of cheap talk, let's get to the business!

Errata for my last post
-----------------------

[Frank Ch. Eigler](http://web.elastic.org/~fche/blog2/), one of
SystemTap's maintainers, kindly mentioned something that I should say
about SystemTap userspace probes.

Basically, it should be clear that `SDT` probes are not the only kind of
userspace probing one can do with SystemTap. There is yet another kind
of probe (maybe even more powerful, depending on the goals):
**DWARF-based function/statement probes**. SystemTap supports this kind
of probing mechanism for quite a while now.

It is not the goal of this post to explain it in detail, but you might
want to give it a try by compiling your binary with debuginfo support
(use the `-g` flag on `GCC`), and do something like:

    #!bash
	$ stap -e 'probe process("/bin/foo").function("name") { log($$parms) }' -c /bin/foo
	$ stap -e 'probe process("/bin/foo").statement("*@file.c:443") { log($$vars) }' -c /bin/foo

And that's it. You can read SystemTap's documentation, or
[this](http://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps)
guide to learn how to add userspace probes.

Using GDB with SystemTap SDT Probes
-----------------------------------

Well, now let's get to the interesting part. It is time to make `GDB`
work with the `SDT` probe that we have put in our example code. Let's
remember it:

    #!c
	#include <sys/sdt.h>

	int
	main (int argc, char *argv[])
	{
	  int a = 10;

	  STAP_PROBE1 (test_program, my_probe, a);

	  return 0;
	}

It is a very simple example, and we will have to extend it later in
order to show more features. But for now, it will do.

The first thing to do is to open `GDB` (with SystemTap support, of
course!), and check to see if it can actually see probe inserted in our
example.

	#!bash
	$ gdb ./test_program
	GNU gdb (GDB) 7.5.50.20121014-cvs
	Copyright (C) 2012 Free Software Foundation, Inc.
	License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
	...
	(gdb) info probes
	Provider     Name     Where              Semaphore Object
	test_program my_probe 0x00000000004004ae           /home/sergio/work/src/git/build/gdb/test_program

Wow, it actually works! :-)

If you have seen something like the above, it means your `GDB` is
correctly recognizing `SDT` probes. If you see an error, or if your
`GDB` doesn't have the `info probes` command, then you'd better make
sure you have a recent version of `GDB` otherwise you won't be able to
use the `SDT` support.

### Putting breakpoints in the code

Anyway, now it is time to start using this support. The first thing I
want to show you is how to put a breakpoint in a probe.

	#!bash
	(gdb) break -probe-stap my_probe
	Breakpoint 1 at 0x4004ae

That's all! We have chosen to extend the `break` command in order to
support the new `-probe-stap` parameter. If you're wondering *... why
the -probe prefix?*, it is because I was asked to implement a complete
abstraction layer inside `GDB` in order to allow more types of probes to
be added in the future. So, for example, if someone implements support
for an hypothetical type of probe called `xyz`, you would have
`break -probe-xyz`. It took me a little more time to implement this
layer, but it is worth the effort.

Anyway, as you have see above, `GDB` recognize the probe's name and
correctly put a breakpoint in it. You can also confirm that it has done
the right thing by matching the address reported by `info probes` with
the one reported by `break`: they should be the same.

Ok, so now, with our `breakpoint` in place, let's run the program and
see what happens.

	#!bash
	(gdb) run
	Starting program: /home/sergio/work/src/git/build/gdb/test_program

	Breakpoint 1, main (argc=1, argv=0x7fffffffdf68) at /tmp/example-stap.c:8
	8  STAP_PROBE1 (test_program, my_probe, a);

As you can see, `GDB` stopped at the exact location of the probe.
Therefore, you are now able to put marks (i.e., probes) in your source
code which are **location-independent**. It means that it doesn't really
matter where in the source code your probe is, and it also doesn't
matter if you change the code around it, changing the line numbers, or
even moving it to another file. `GDB` will always find your probe, and
always stop at the right location. Neat!

### Examining probes' arguments

But wait, there's more! Remember when I told you that you could also
inspect the probe's arguments? Yes, let's do it now!

Just remember that, in `SDT`'s parlance, the current probe's argument is
`a`. So let's print its value.

	#!bash
	(gdb) p $_probe_arg0
	$1 = 10
	(gdb) p a
	$2 = 10

*"Hey, captain, it seems the boat really floats!"*

Check the source code above, and convince yourself that `a`'s value is
`10` :-). As you might have seen, I have used a fairly strange way of
printing it. It is because the probe's arguments are available inside
`GDB` by means of **convenience variables**. You can see a list of them
[here](http://sourceware.org/gdb/current/onlinedocs/gdb/Convenience-Vars.html#Convenience-Vars).

Since `SDT` probes can have up to 12 arguments (i.e., you can use
`STAP_PROBE1` ... `STAP_PROBE12`), we have created inside `GDB` 12
convenience variables, named `$_probe_arg0` until `$_probe_arg11`. I
know, it is not an easy name to remember, and even the relation between
`SDT` naming and `GDB` naming is not direct (i.e., you have to subtract
1 from the `SDT` probe number). If you are not satisfied with this,
please open a bug in our [bugzilla](http://sourceware.org/bugzilla/) and
I promise we will discuss other options.

I would like to emphasize something here: just as you don't need
debuginfo support for dealing with probes inside `GDB`, you also don't
need debuginfo support for dealing with their arguments as well. It
means that you can actually compile your code without debuginfo support,
but still have access to some important variables/expressions when
debugging it. Depending on how `GCC` optimizes your code, you may
experience some difficulties with argument printing, but so far I
haven't heard of anything like that.

More to come
------------

Ok, now we have covered more things about the `SDT` probe support inside
`GDB`, and I hope you understood all the concepts. It is not hard to get
things going with this, specially because you don't need extra libraries
to make it work.

In the next post, I intend to finish this series by explaining how to
use `tracepoints` with `SDT` probes. Also, as I said in the previous
post of this series, maybe I will talk a little bit about how the `SDT`
probes are organized within the binary.

See you soon!