Back to Oh My Openagent

Native Binary Debugging (No Source / Reverse Engineering)

packages/omo-codex/plugin/skills/debugging/references/runtimes/native-binary.md

4.5.1220.9 KB
Original Source

Native Binary Debugging (No Source / Reverse Engineering)

For binaries where you don't have trustworthy source: stripped production builds, third-party closed libs, malware, CTF challenges, firmware, vendored libs whose docs lie. The workflow is specific; doing it out of order wastes days.

This reference coordinates the triage and dynamic work. The heavy tools each have their own reference:

Read those before using them — especially Ghidra, which has a surprising amount of workflow that's not obvious.


⚠️ STOP — is this actually a stripped C/C++ binary?

A growing share of "binaries" are actually bundled high-level apps — Bun SEA, Node SEA, Deno compile, pkg, nexe, Electron, Tauri, PyInstaller. Their workflow is completely different: the high-level source is recoverable with the right per-bundler tool (often plaintext, sometimes V8 cache / .pyc / eszip needing extra tooling), and Ghidra against the runtime VM wastes hours.

Quick check:

bash
file ./target                                                    # Mach-O / ELF / PE - inconclusive
du -h ./target                                                   # 50 MB+ for a "simple CLI" → suspect bundled
strings -n 12 ./target | rg -iE 'bun|node_modules|webpack|esbuild|deno|pkg/lib|electron|pyinstaller|nexe|NODE_SEA_FUSE|tauri' | head -5

If any hits → close this file, open bundled-js-binary.md instead. Following the Ghidra/pwndbg path on a bundled-app binary wastes hours decompiling the runtime VM while the app-level bundle is recoverable with the right per-bundler tool (plaintext for Bun/pkg/nexe/Electron-asar; eszip / V8-cache / .pyc for Deno / Node SEA / PyInstaller).

If file says "Mach-O" or "ELF", du is < 20 MB, and the strings check is empty → continue here.


The workflow (do these in order)

Every step's output is input to the next. Skipping steps means guessing later.

  [1] Triage           →  what kind of binary is this?
  [2] Dynamic tracing  →  what syscalls / libcalls does it make?
  [3] Static analysis  →  what does it DO, in readable form? (Ghidra)
  [4] Dynamic debug    →  confirm hypotheses at runtime (pwndbg)
  [5] Scripted repro   →  lock the bug with a pwntools script
  [6] TDD + fix / report

Steps 1 and 2 are fast (minutes). Step 3 is slow (tens of minutes to hours depending on size). Don't skip 1-2 and go straight to Ghidra — the triage output tells you what to focus on inside Ghidra.


[1] Triage — 5-minute fingerprint

bash
# Basic identity
file ./target
#   elf, mach-o, pe? 32/64-bit? dynamically linked? stripped?

# Architecture details
readelf -h ./target                    # ELF header: entry point, arch, type
lipo -info ./target 2>/dev/null        # macOS: universal binary?

# Interesting strings (often leaks function names, error messages, URLs, API keys)
strings -n 8 ./target | head -100
strings -n 8 ./target | grep -iE '(http|/api/|error|debug|version)'

# Imported symbols (what does it link against?)
nm -D ./target 2>/dev/null              # dynamic symbols
objdump -T ./target 2>/dev/null         # same, alternate tool
readelf -d ./target                     # dynamic section (NEEDED libs)
ldd ./target 2>/dev/null                # resolved library paths

# Security posture (affects what exploits / bugs are possible)
checksec --file=./target                # requires pwntools or installing checksec
#   NX, PIE, RELRO, stack canary, FORTIFY

# Is it stripped?
nm ./target 2>/dev/null | head          # empty? stripped. full? not stripped.
file ./target                           # will say "stripped" or "not stripped"

⚠️ strings -n N silently drops short content

strings prints runs of printable characters of length ≥ N. With -n 8, anything shorter than 8 chars sandwiched between non-printable bytes is dropped silently. This includes:

  • Short identifier interpolations in templates (${x}, ${i}, ${R})
  • Short embedded constants (v3, null, integer immediates as bytes)
  • Short error codes between binary padding

Real example: a JavaScript template literal <INSTRUCTIONS>\n${x}\n</INSTRUCTIONS> came out of strings -n 8 as <INSTRUCTIONS>\n</INSTRUCTIONS> — the ${x} (4 chars) was dropped. A consumer reading the dump would conclude the template was empty. It is not.

Use strings only for fingerprinting (Phase 1). For any extraction whose correctness matters, read bytes directly:

bash
# Count occurrences of a needle
LC_ALL=C grep -aoc 'NEEDLE' ./target

# Find offsets
LC_ALL=C grep -aob 'NEEDLE' ./target | head

# Or via Python for byte-precise context
python3 -c "
import sys
data = open('./target','rb').read()
needle = b'NEEDLE'
pos = data.find(needle)
print(repr(data[max(0,pos-100):pos+200]))
"

If you must keep using strings, lower the threshold: strings -n 1 -t x ./target | rg .... The signal-to-noise drops sharply but short content is preserved.

Write the triage summary to the journal:

markdown
## Binary triage
- Type: <ELF 64-bit, dynamically linked, stripped>
- Arch: <x86_64 | arm64 | ...>
- Libs: <libc, openssl, libcurl>
- Security: <NX, PIE, Partial RELRO, no canary>
- Interesting strings: <short list>
- First hypothesis surface: <which function / area looks most relevant>

[2] Dynamic tracing — what does it actually call?

These are cheap — run them before Ghidra to orient yourself.

Linux: strace + ltrace

bash
# System calls
strace -f -o trace.out ./target arg1 arg2
strace -f -e trace=network ./target           # filter to network syscalls
strace -f -e trace=file ./target              # filter to file ops

# Library calls (less useful when stripped but still informative)
ltrace -f -o ltrace.out ./target
ltrace -f -e 'str*+mem*' ./target             # filter to string/mem functions

macOS: Mach-O specifics

SIP block reality check. With System Integrity Protection enabled (default on every modern macOS), dtruss / dtrace will silently fail to attach to:

  • Anything in /usr, /bin, /sbin, /System
  • Apple-signed binaries (Xcode CLT, Homebrew formulae from Apple-distributed taps)
  • Notarized vendor binaries (Bun, Deno, Docker Desktop, etc.)

dtruss ./target will appear to run but produce zero events. This is not a bug; it is the SIP design. Disabling SIP requires a Recovery Mode reboot — usually not worth it. Use the alternatives below.

bash
# dtruss — works only when SIP allows it (your own unsigned binaries)
sudo dtruss -f ./target 2>&1 | head -20         # equivalent to strace
# If output is suspiciously empty → SIP blocked it. Switch to lldb or app-level logging.

Mach-O metadata inspection (no SIP issues, no debugger needed):

bash
# Architecture and slices
file ./target                                    # arm64 / x86_64 / universal
lipo -info ./target                              # which architectures included
lipo -thin arm64 ./target -output ./target-arm64 # extract one slice for analysis

# Headers & load commands (segments, dylibs, code-signature pointer)
otool -h ./target                                # Mach header (cputype, ncmds, flags)
otool -l ./target | head -100                    # load commands; entitlements live in code-signature blob, see codesign below

# Dynamic library dependencies (macOS equivalent of ldd)
otool -L ./target                                # linked dylibs with versions
dyld_info ./target                               # macOS 13+, more detailed than otool -L

# Disassembly
otool -tv ./target | head -200                   # quick disassembly without Ghidra
otool -tV ./target                               # with symbol-resolved branches

# Imported / exported symbols (Apple `nm`, NOT GNU)
nm -u ./target                                   # undefined references = imports
nm -gU ./target                                  # external defined = exports
# Note: GNU `-D`/dynamic flags are not honored on Apple `nm`; use the above forms.
symbols -fullSourcePath -onlyWithDebugInfo ./target  # if any debug info survives

# Code signature & entitlements (entitlements come from codesign, NOT otool)
codesign -dv --entitlements :- ./target 2>&1     # signature info + entitlements XML on stdout
spctl --assess --type execute -vv ./target       # Gatekeeper assessment

# Cert chain — extract to a temp dir to avoid creating files named -0/-1 in cwd
tmp=$(mktemp -d)
codesign -dvv --extract-certificates="$tmp/cert" ./target 2>&1
ls -la "$tmp"
# rm -rf "$tmp"  # journal first, clean up later

# Strings inside specific segments only (less noise than full-binary strings)
otool -s __TEXT __cstring ./target               # C string section
otool -s __TEXT __const ./target                 # constants section

Interactive debugging on macOS — use lldb, not gdb.

GDB on macOS requires a self-signed code-signing certificate (codesign --entitlements gdb.entitlements --sign gdb-cert /opt/homebrew/bin/gdb) and even then is unreliable on arm64. Use lldb directly — it ships with Xcode CLT and works without configuration.

bash
# Start lldb
lldb ./target

# Set arguments
(lldb) settings set target.run-args arg1 arg2

# Run with breakpoints
(lldb) breakpoint set --name function_name        # symbol-based
(lldb) breakpoint set --address 0x1000034c0       # address-based
(lldb) breakpoint set --regex '.*decode.*'         # regex over symbols

# Run / step / inspect
(lldb) run
(lldb) bt                                          # backtrace
(lldb) frame variable                              # locals
(lldb) register read                               # all registers
(lldb) memory read --size 8 --format x --count 16 $sp   # 16 qwords from stack
(lldb) disassemble --frame                         # current function
(lldb) image list                                  # loaded modules
(lldb) image lookup -a 0x1000034c0                 # which module + symbol owns this address

# Process attach to running process
(lldb) process attach --pid 12345
(lldb) process attach --name target               # attach by name

# Print Mach-O specific
(lldb) image dump sections ./target
(lldb) image dump symtab ./target

Function interception via DYLD_INSERT_LIBRARIES (macOS equivalent of LD_PRELOAD):

bash
# Build a shim dylib that overrides specific functions
# Then run target with it preloaded
DYLD_INSERT_LIBRARIES=./shim.dylib DYLD_FORCE_FLAT_NAMESPACE=1 ./target

DYLD_INSERT works in the unrestricted case but is blocked in three distinct scenarios — distinguish them when diagnosing why your shim didn't load:

  1. SIP / restricted process (target has the __RESTRICT,__restrict section, is setuid/setgid, or is a platform/Apple-signed binary): dyld unconditionally strips all DYLD_* env vars before the process starts. Nothing you set will reach the target.
  2. Hardened runtime + library validation (CS_RUNTIME flag set, com.apple.security.cs.disable-library-validation entitlement absent): the process accepts DYLD_INSERT_LIBRARIES but rejects loading any dylib that isn't signed by the same Team ID or by Apple. Symptom: shim is found but not loaded; check log show --predicate 'eventMessage CONTAINS "library validation failed"'.
  3. Notarization / Gatekeeper translocation: the binary may be running from a translocated path; relative paths in DYLD_INSERT_LIBRARIES won't resolve. Use absolute paths.

Check each:

bash
# Restrict segment present? (case 1)
otool -l ./target | grep -A2 __RESTRICT
# Hardened runtime flag? (case 2)
codesign -d --verbose=4 ./target 2>&1 | grep -iE 'flags=|CodeDirectory'
# Look for "0x10000(runtime)" or similar in the flags line.
# Disable-library-validation entitlement?
codesign -d --entitlements :- ./target 2>&1 | grep disable-library-validation

App-level debug logging (always works, ignores SIP):

When debugger attach is blocked, fall back to maximizing the app's own logging:

bash
# Try common patterns
APP_DEBUG=1 APP_LOG_LEVEL=debug APP_LOG_FILE=/tmp/trace.log ./target
NSDebugEnabled=YES ./target                       # Cocoa apps
OS_ACTIVITY_MODE=debug ./target                   # os_log subsystem

# Then read os_log unified logging stream live
log stream --predicate 'process == "target"' --level debug

# Or extract historical logs
log show --predicate 'process == "target"' --last 1h --info --debug

This is the partial-runtime-evidence path for macOS. See methodology/partial-runtime-evidence.md for how to combine app-level logs with static analysis when wire-level capture is blocked.

Network capture on macOS (TLS-decrypted):

bash
# 1. Find the active network service (don't assume "Wi-Fi"):
#    Map the default-route interface to the matching networksetup service name.
networksetup -listallnetworkservices                        # show options
DEFAULT_IF=$(route -n get default 2>/dev/null | awk '/interface:/ {print $2}')
echo "Default-route interface: $DEFAULT_IF"
# Match the interface (en0, en1, ...) back to a service name:
SERVICE=$(networksetup -listallhardwareports | awk -v iface="$DEFAULT_IF" '
  /^Hardware Port:/ { hp = substr($0, index($0,$3)) }
  /^Device:/        { if ($2 == iface) print hp }
')
if [ -z "$SERVICE" ]; then
  echo "Could not auto-detect active service. Pick one from -listallnetworkservices manually." >&2
  echo "Aborting proxy setup." >&2
  false  # signal failure but stay safe at top level
else
  echo "Using service: $SERVICE"
fi

# 2. JOURNAL the original proxy state before changing it (REQUIRED for safe rollback):
networksetup -getwebproxy "$SERVICE"        # save this output to journal
networksetup -getsecurewebproxy "$SERVICE"  # save this too

# 3. Start mitmproxy with persistent CA at ~/.mitmproxy/
mitmproxy --listen-host 127.0.0.1 --listen-port 8888 &

# 4. Trust the mitmproxy CA system-wide if the target uses URLSession or any framework
# that ignores HTTPS_PROXY/SSL_CERT_FILE (most macOS-native apps do):
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/.mitmproxy/mitmproxy-ca-cert.pem

# 5. Two routing options. Try env-var first; fall back to system proxy:
# 5a. Apps that honor env vars (most CLIs):
HTTPS_PROXY=http://127.0.0.1:8888 SSL_CERT_FILE=~/.mitmproxy/mitmproxy-ca-cert.pem ./target ...

# 5b. Apps that use URLSession / system network config (most GUI apps, Bun, some CLIs):
networksetup -setwebproxy "$SERVICE" 127.0.0.1 8888
networksetup -setsecurewebproxy "$SERVICE" 127.0.0.1 8888

# 6. Cleanup — RESTORE original state from journal, untrust CA:
networksetup -setwebproxystate "$SERVICE" off
networksetup -setsecurewebproxystate "$SERVICE" off
sudo security delete-certificate -c "mitmproxy" /Library/Keychains/System.keychain

Critical: forgetting step 6 leaves all your subsequent traffic mis-routed and silently MITM-able. Journal every step.

What to look for

ObservationHypothesis
open("/etc/secret-config", ...)Reads unexpected config; look at what it does with contents
connect(... 1.2.3.4:443)Phones home or depends on an external service
getenv("FOO") returning NULLEnv var expected but not set
Repeated poll/epoll_wait with no progressStuck on I/O; check downstream
SIGSEGV caught by signal handlerCustom crash recovery — often hides the real bug
dlopen("libfoo.so.42")Dynamic plugin loading; check plugin path

[3] Static analysis with Ghidra

When triage + tracing have narrowed you to "something in function X" or "the crypto routine is weird", open Ghidra.

Open tools/ghidra.md before launching Ghidra — the import / analyze / decompile workflow is not obvious and first-time users waste an hour figuring it out.

Ghidra's decompiler turns machine code into readable-ish C. That's usually what you want. Stay in the Decompiler view; drop to Listing (disassembly) only when the decompiler punts.


[4] Dynamic debugging with pwndbg

Once static analysis gives you a hypothesis ("this branch at 0x401234 is where the validation fails"), confirm it at runtime with pwndbg.

Open tools/pwndbg.md before launching gdb. Pwndbg gives you the context view (registers / stack / disasm / code all visible at once) which is essential for binary debugging.

Typical pwndbg flow:

$ gdb ./target                                 # pwndbg loads automatically if installed
pwndbg> break *0x401234                        # break at the address static analysis flagged
pwndbg> run arg1 arg2
# At the breakpoint:
pwndbg> context                                # registers + stack + disasm
pwndbg> telescope $rdi                         # walk pointers at $rdi
pwndbg> x/20xw $rsp                            # raw dump of stack
pwndbg> ni / si                                # step next / step instruction

[5] Scripted reproduction with pwntools

Once you have a hypothesis with a concrete repro input, lock it down with pwntools. This is the "failing test" equivalent for binaries.

Open tools/pwntools.md — the Process/Remote/ELF/context APIs are the foundation.

python
from pwn import *

context.binary = elf = ELF('./target')

p = process('./target')
p.sendlineafter(b'> ', b'<trigger input that reproduces the bug>')
result = p.recvall(timeout=3)
assert b'expected-output-when-fixed' in result, f'bug repro: {result}'

This script is now your "red test". When the fix is applied, the script should pass (or the assertion should be inverted for negative tests — e.g. "the crash string should NOT appear").


[6] Fixing a binary bug you can't recompile

Three options, in preference order:

Option A: Patch at the source (if you have it)

If the bug is in your own code and source is available, fix it there and rebuild. Standard TDD path.

Option B: Binary patch

For tiny fixes (one byte, one branch inversion):

bash
# Identify the exact byte offset
# e.g. Ghidra says the bug is at 0x401234 = file offset 0x1234
printf '\x90\x90' | dd of=./target bs=1 seek=$((0x1234)) conv=notrunc

Journal the exact dd command and the original bytes so you can revert.

Option C: Wrap / shim

If you can't patch the binary, write a shim library (LD_PRELOAD on Linux, DYLD_INSERT_LIBRARIES on macOS) that overrides the buggy function. pwntools has examples.

Option D: Report upstream

If it's a third-party binary and none of the above are feasible, the "fix" is a high-quality bug report with:

  • Full triage summary
  • Reproducible pwntools script
  • Ghidra decompilation of the buggy function
  • Hypothesis about the root cause
  • Recommended patch sketch (in C or pseudocode)

Silent-failure patterns in native binaries

PatternWhy it's silent
Ignored libc return codes (read, write, malloc)Bug continues with garbage data; no check
Signal handler swallows SIGSEGVCrash converted to "something didn't work"; no log
setjmp/longjmp unwinding over cleanupResources leak silently
Thread-local error state never read (errno, GetLastError)Error happened, nobody asked
Recovered assertion failure in release buildassert compiled out; precondition violations silently corrupt
Dangling pointer reads after freeOften looks like valid data until it doesn't

Phase 9 cleanup specifics

bash
# Kill debugger sessions
pkill -f 'gdb' || true
pkill -f 'lldb' || true

# Ghidra scratch projects (if made just for this session)
# Named something like ~/ghidra-projects/debug-<timestamp>:
ls -la ~/ghidra-projects/ 2>/dev/null
# rm -rf ~/ghidra-projects/debug-scratch         # only if the journal says to

# Core dumps left from crashes
rm -f ./core ./core.* ~/core.*

# strace/ltrace output files
rm -f trace.out ltrace.out

# If you made a binary patch (Option B above), confirm revert
# The journal should have the original bytes — restore them:
# printf '<original-bytes>' | dd of=./target bs=1 seek=<offset> conv=notrunc

# Trace-output files
rm -f /tmp/debug-*.bin /tmp/debug-*.strace /tmp/debug-*.ltrace

# macOS-specific:
# Restore proxy settings if you set them (CRITICAL — leaves system traffic mis-routed otherwise)
# Use the SAME $SERVICE you used when enabling the proxy (read it from the journal).
# Do NOT hardcode "Wi-Fi" — many machines route traffic over Ethernet, USB tether, or a VPN service.
[ -n "$SERVICE" ] && {
  networksetup -setwebproxystate "$SERVICE" off 2>/dev/null
  networksetup -setsecurewebproxystate "$SERVICE" off 2>/dev/null
}
# Or restore explicitly from the journaled original state — see the proxy section above.

# Stop mitmproxy
pkill -f 'mitmproxy' 2>/dev/null

# Remove DYLD shim libraries you built
rm -f /tmp/*-shim.dylib

# Clear extracted strings dumps (these can be huge and may contain secrets)
rm -f /tmp/*-strings*.txt

# Verify hostname resolution returns to normal (mitmproxy can leave entries)
scutil --dns | head -20