systemtap: a linux trace/probe tool

Visit the project web site at <http://sourceware.org/systemtap>,
for documentation and mailing lists for developers and users.

This is free software.
See the COPYING file for redistribution/modification terms.
See the INSTALL file for generic build instructions.
See the HACKING file for contribution advice.

Prerequisites:

- linux kernel
- kernel module build environment (kernel-devel rpm) and/or dyninst
- optionally, debugging information for kernel/user-space being instrumented
- C compiler (same as what kernel was compiled with)
- elfutils with libdwfl for debugging information parsing
- root privileges

Installation steps:

- Install any debuginfo packages you need, for kernel and/or userspace.
  On modern Fedora, # debuginfo-install kernel [...]
  (Beware of confusion between kernel vs. kernel-debug vs kernel-PAE etc.
  variants.  Each likely has a corresponding development and debuginfo
  package.)

- Install the systemtap package.
  On modern Fedora, # yum install systemtap systemtap-runtime

Build steps:

- Consider installing the kernel-debuginfo, kernel-devel, gcc and
  dependent packages (or see below if you are building your own
  kernels from source).  If using only the pure-userspace dyninst
  backend, install gcc and dyninst-devel.
  
- If available, install your distribution's copy of elfutils and its
  development headers/libraries.
  Or if desired, download an elfutils source release to build in
  "bundled mode" (below), and untar it into some new directory.
  Or if desired, build elfutils separately one time, and install
  it to /usr/local.
  See http://fedorahosted.org/elfutils/
  Version 0.151 is recommended for i386 hosts probing prelinked programs.
  (PR12141)

- On modern Fedora, install general optional build-requisites:
  # yum-builddep systemtap
  On modern Debian/Ubuntu, similarly:
  # apt-get build-dep systemtap

- Download systemtap sources:
  http://sourceware.org/systemtap/ftp/releases/
  http://sourceware.org/systemtap/ftp/snapshots/
  (or)
  git clone git://sourceware.org/git/systemtap.git
      (or) http://sourceware.org/git/systemtap.git

- Build systemtap normally:
  % .../configure [other autoconf options]
  Or, with build it with a bundled internal copy of elfutils:
  % .../configure --with-elfutils=ELFUTILS-SOURCE-DIR [other autoconf options]
  (Note that elfutils > 0.139 requires gcc > 4.0 or else the
  appropriate elfutils-portability.patch.  Ensure decompression
  library headers/libraries are installed for elfutils' use.)

  Consider configuring with "--enable-dejazilla" to automatically
  contribute to our public test result database.

  Consider configuring with "--prefix=DIRECTORY" to specify an
  installation directory other than /usr/local.  It can be an ordinary
  personal directory.

  % make all
  # make install

  To uninstall systemtap:
  # make uninstall

- Run systemtap:

  To run systemtap after installation, add $prefix/bin to your $PATH, or
  refer to $prefix/bin/stap directly.  If you keep your build tree
  around, you can also use the "stap" binary there.  

  Some samples should be available under $prefix/share/doc/systemtap/examples.

  For the normal linux-kernel-module based backend, run "stap" as
  root.  If desired, create "stapdev" and "stapusr" entries in
  /etc/groups.  Any users in "stapdev"+"stapusr" will be able to run
  systemtap as if with root privileges.  Users in "stapusr" only may
  launch (with "staprun") pre-compiled probe modules (created by "stap
  -p4 ...") that a system administrator copied under
  /lib/modules/`uname -r`/systemtap.  "stapusr" may also be permitted
  to create arbitrary unprivileged systemtap scripts of their own.
  See README.unprivileged for additional setup instructions.

  To run a simple test.
  # stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

  To run the full test suite from the build tree.
  # make installcheck

  For the prototype dyninst pure-userspace backend, run "stap" as any user.
  % stap --runtime=dyninst -e 'probe process.function("*") { 
                               println(pn(), ":", $$parms) }' -c 'ls'


Tips:

- By default, systemtap looks for the debug info in these locations:
  /boot/vmlinux-`uname -r`
  /usr/lib/debug/lib/modules/`uname -r`/vmlinux
  /lib/modules/`uname -r`/vmlinux
  /lib/modules/`uname -r`/build/vmlinux


Building a kernel.org kernel:

- Consider applying the utrace kernel patches, if you wish to probe
  user-space applications.  http://sourceware.org/systemtap/wiki/utrace
  Or if your kernel is near 3.5, apply the uprobes and related patches
  (see NEWS).  Or if your kernel is >= 3.5, enjoy the built-in uprobes.

- Build the kernel using your normal procedures.  Enable
  CONFIG_DEBUG_INFO, CONFIG_KPROBES, CONFIG_RELAY, CONFIG_DEBUG_FS,
  CONFIG_MODULES, CONFIG_MODULE_UNLOAD, CONFIG_UTRACE if able
- % make modules_install install headers_install
- Boot into the kernel.

- If you wish to leave the kernel build tree in place, simply run
  % stap -r /path/to/kernel/build/tree [...]
  You're done.

- Or else, if you wish to install the kernel build/debuginfo data into
  a place where systemtap will find it without the "-r" option:
  % ln -s /path/to/kernel/build/tree /lib/modules/RELEASE/build 

- Instead of using the "-r" option, you can also use the environment 
  variable SYSTEMTAP_RELEASE to direct systemtap to the kernel data.