dbgsourcer: Fetch source code for gdb/lldb using an arbitrary command

Introduction

dbgsourcer is is a Linux LD_PRELOAD library to simulate a mounted filesystem in order to provide source files to gdb or lldb that are provided by an arbitrary command, so they do not need to be real files in the Linux host's filesystem.

Motivation

dbgsourcer is to enable gdb or lldb to fetch source files that are not available as real files in the Linux host's filesystem and when you cannot use other methods such as FUSE or copying all the source files. For example:

1. You want to fetch the source files from another host via ssh.
2. You want to fetch the source files from a local git repository from an arbitrary (and not currently checked-out) commit.

Notes:

• If you are able to use the user space file system - FUSE - then it would probably be better to use sshfs for (1) and gitfs/repofs for (2).

• If you are able to copy all the source files, for example using rsync, that would probably be as simple if not simpler. But in some cases it might be too slow or too big because the source tree is large.

• It would be better if gdb and lldb could natively fetch files from an arbitrary command.

• There is no reason why this library should only work for these debuggers. However, I have not tested it with other applications and it has some limitations, listed below.

Environment variables

You must define these environment variables:

Set $DBGSOURCER_MOUNTPOINT to the simulated "mount point". It must be an existing directory. Tell gdb or lldb to look for source files under this path.

Set $DBGSOURCER_COMMAND to a printf-style string for the command that dbgsourcer will run to fetch a file from a given path in the simulated filesystem to a destination file path in the temporary cached directory. dbgsourcer will replace "%S" in the command string with the path of the file relative to the mountpoint; "%D" with the destination file path; and "%%" with a single "%". In order for stat() to report the correct modification (and access) time, the command should also set the time on the destination file. This is necessary for the gdb check that the source file is older than the binary to work correctly (otherwise it will print a warning). If the command returns a non-zero exit status, dbgsourcer will report that the file does not exist.

Possible commands to fetch source files and set the modification time include:

• scp -p user@host:/path/to/%S %D to copy a remote file over ssh.
• git-getfile /path/to/git-repo git-reference %S %Dto fetch files from agit`repository at the given reference.
• cp -p /path/to/%S %D to copy a local file.

These variables are optional:

• Define $DBGSOURCER_TEMPDIR to the location where dbgsourcer will cache the files it fetches. The default is /tmp.
• Define $DBGSOURCER_DEBUG to enable debug output.
• Define $DBGSOURCER_TARGET_PROCESS_NAME to the name of the process that you want dbgsourcer to target, while not affecting other processes. This can be useful if you want to run the debugger from a wrapper script. If not set then dbgsourcer will target the initial process.

Examples

# Setup: Compile dbgsourcer and the demo program.
# The demo program is compiled with `-fdebug-prefix-map` to "hide" the
# source code, so the debugger will not find it without help and this
# is a real test.
cd /path/to/dbgsourcer
make && (cd test/demo && make)

# Setup: Make a "mount point".
mkdir /tmp/my_mount_point

# Debug the demo program with gdb.
LD_PRELOAD="./dbgsourcer.so" \
    DBGSOURCER_MOUNTPOINT="/tmp/my_mount_point" \
    DBGSOURCER_COMMAND="cp -a test/demo/src/%S %D" \
    gdb test/demo/build/testprog \
        --ex "set pagination off" \
        --ex "set substitute-path not-src /tmp/my_mount_point" \
        --ex "list main" \
        --ex "quit"

# Debug the demo program with lldb.
LD_PRELOAD="./dbgsourcer.so" \
    DBGSOURCER_MOUNTPOINT="/tmp/my_mount_point" \
    DBGSOURCER_COMMAND="cp -a test/demo/src/%S %D" \
    lldb test/demo/build/testprog \
        -o "settings set target.source-map \
            $(realpath $(pwd))/test/demo/not-src \
            /tmp/my_mount_point" \
        -o "list main" \
        -o "quit"

# Debug `my_binary` with gdb, fetching sources from user `me` on the
# server `myhost` under the directory `/path/to/source`.
LD_PRELOAD="./dbgsourcer.so" \
    DBGSOURCER_MOUNTPOINT="/tmp/my_mount_point" \
    DBGSOURCER_COMMAND="scp -p me@myhost:/path/to/source/%S %D" \
    gdb ./my_binary \
        --ex "set substitute-path /path/to/source /tmp/my_mount_point" \
        --ex "list my_function" \
        --ex "quit"

# Debug `my_binary` with lldb, fetching sources from the reference `V1.2`
# in the git repository in `/path/to/git-repo`.
LD_PRELOAD="./dbgsourcer.so" \
    DBGSOURCER_MOUNTPOINT="/tmp/my_mount_point" \
    DBGSOURCER_COMMAND="./git-getfile /path/to/git-repo V1.2 %S %D" \
    lldb ./my_binary \
        -o "settings set target.source-map /path/to/source \
                                           /tmp/my_mount_point" \
        -o "list my_function" \
        -o "quit"

How it works

dbgsourcer works by intercepting stat() and open() calls under a "mount point" given by $DBGSOURCER_MOUNTPOINT; it runs the command given by $DBGSOURCER_COMMAND to fetch a copy of the file, relative to the mount point. It saves the file in a temporary directory under $DBGSOURCER_TEMPDIR (or /tmp) and calls stat() or open() on that cached copy. It used the cached copy for later accesses.

Limitations

Limitations of the implementation:

• Only allows open() with the O_RDONLY option and a few flags.
• Only intercepts open() and stat().
• Does not intercept openat(), the other stat functions or any other functions.
• Fetches each file and saves a copy in the temporary directory and does not fetch it again.
  • If the "real" version of the file changes (e.g. you updated it on a remote filesystem) then the program will not see the change. Workaround: delete the contents of the temporary directory to force dbgsourcer to fetch the files again. This will work for files for which the program does not currently hold open file descriptors.
  • If the program calls open() or stat() on lots of files then the temporary directory will become large.
• Only implements one mount point. But you can have your script interpret subdirectories under the mount point as indicators for different actions.
• It checks the path of every open() and stat() call and will therefore make performance
• It is less fun than https://github.com/GregTheMadMonk/libemotify-stderr

LD_PRELOAD libraries in general will not be able to intercept the following:

• mmap()
• syscalls and statically linked standard libraries, e.g. glibc
• setuid binaries (which ignore LD_PRELOAD)
• functions such as openat2 which do not have a glibc wrapper
• newer versions of libraries which change their internals (e.g. in header files) in one of the above ways and stop working

Credits

I initially wrote this library using ldpfuse by Sjors Holtrop: see https://github.com/sholtrop/ldpfuse. I later realised that I only needed to intercept a couple of functions and I ended up reimplementing most of the code.

I copied a small amount of the code from ldpfuse and the list of LD_PRELOAD limitations above. I have retained Sjors Holtrop's copyright notice and choice of the MIT License.

License and copyright

MIT License

Copyright (c) 2025 Ben Cohen

Copyright (c) 2022 Sjors Holtrop