Error allocating remote code buffer when instrumenting AppleGVA / AppleVPA / RawCamera on macOS x86_64

[Super-Code774]

Hello, here is the issue I am having while using the Jacklope fuzzer with TinyInst on macOS.
OS: macOS 26.4.1 (x86_64)

Command used:

./fuzzer -mute_child -in hevc_in -out hevc_out -t 1000 \
-delivery_dir /Volumes/RAMDisk -file_extension mov \
-instrument_module AppleGVA \
-target_module vtdecode -target_method _fuzz \
-nargs 1 -iterations 5000 -persist -loop -cmp_coverage \
-- ./examples/VideoToolbox/vtdecode @@

Error output:

Fuzzer version 1.00
10 input files read
Running input sample hevc_in/IMG_0966 2.MOV
[-] PROGRAM ABORT : Error allocating remote code buffer
       Location : InstrumentModule(), tinyinst.cpp:952

The fuzzer aborts immediately when attempting to instrument certain Apple frameworks.
The issue occurs when instrumenting:

• AppleGVA
• AppleVPA
• RawCamera
• AppleIntelKBLGraphicsMTLDriver

Are there any easy fixes for this, or is there something I might be doing wrong?

[ifratric]

Thanks for reporting! I'm travelling at the moment without access to macOS hardware, so I'll take a look when back in the office.

Possible reason: on x86/x64, TinyInst allocates a code buffer within 2GB range of the module to be instrumented, which makes instrumentation easier. It is possible that dyld shared cache has grown so much that finding free space in this region has become difficult on the latest macOS. There might be optimizations we can do, but I'd have to be able to reproduce the issue and experiment. If this is the reason, arm64 won't be affected as in arm64 case the code buffer can be allocated anywhere in memory.

[Super-Code774]

Thanks for getting back to me!

I think you’re right about the 2GB range being the problem. Looking at the code, x86_64 uses RemoteAllocateNear() to keep the buffer within ±2GB of the target module, while ARM64 just uses RemoteAllocate() with no proximity requirement:

#ifdef ARM64
    (unsigned char *)RemoteAllocate(data->coverage_buffer_size, READONLY, true);
#else
    (unsigned char *)RemoteAllocateNear(
      (uint64_t)module->instrumented_code_remote,
      (uint64_t)module->instrumented_code_remote +
          module->instrumented_code_size,
      data->coverage_buffer_size, READONLY, true);
#endif

I’ll keep thinking about it on my side and wait until you have access to Mac hardware .
Thanks!