What do you do when you need to see what a program is doing, but it's not one that you'd normally run from the command line? Perhaps it's one that is called as a network daemon from inetd, is called from inside another shell script or application, or is even called from cron. Is it actually being called? What command line parameters is it being handed? Why is it dying?

Let's assume the app in question is /the/path/to/myapp . Here's what you do. Make sure you have the "strace" program installed. Download "apptrace" from ftp://ftp.stearns.org/pub/apptrace/ and place it in your path, mode 755. Then type:

apptrace /the/path/to/myapp

When that program is called in the future, apptrace will record the last time myapp ran (see the timestamp on myapp-last-run), the command line parameters used (see myapp-parameters), and the strace output from running myapp (see myapp.pid.trace) in either $HOME/apptrace or /tmp/apptrace if $HOME is not set.

Note that if the original application is setuid-root, strace will not honor that flag and it will run with the permissions of the user running it like any other non-setuid-root app. See the man page for strace for more information on why.

When you've found out what you need to know and wish to stop monitoring the application, type:

mv -f /the/path/to/myapp.orig /the/path/to/myapp

Many thanks to David S. Miller , kernel hacker extraordinaire, for the right to publish his idea. His original version was:

It's actually pretty easy once if you can get a shell on the machine
before the event, once you know the program in question:

mv /path/to/${PROGRAM} /path/to/${PROGRAM}.ORIG
edit /path/to/${PROGRAM}
#!/bin/sh
strace -f -o /tmp/${PROGRAM}.trace /path/to/${PROGRAM}.ORIG $*

I do it all the time to debug network services started from
inetd for example.

William is an Open-Source developer, enthusiast, and advocate from Vermont, USA.