flac binary instrumentation problem

[De3mond]

• description: When I use the following prompt to perform DynamoRIO instrumentation, everything works fine, but when using TinyInst as the instrumentation mode, there is a crash issue (no crash occurs when executing the seed directly). All information is listed below.

• Env

  • windows server 2019
  • flac: 1.3.3
  • TintInst commit: 9cdc11e

• prompt: litecov.exe -instrument_module flac.exe -target_module flac.exe -target_offset 0xdf70 -nargs 3 -iterations 1 -persist -loop -trace_debug_events -- flac.exe in\sample.flac --force

• output:

Debugger: Process created or attached
Debugger: Exception 80000003 at address 00007FF820BD338C
Debugger: Exception 4000001f at address 0000000076F6F886
Debugger: Loaded module flac.exe at 0000000000860000
Debugger: Loaded module ntdll.dll at 0000000076EC0000
Debugger: Loaded module KERNEL32.DLL at 0000000074FB0000
Debugger: Loaded module KERNELBASE.dll at 0000000075090000
Debugger: Loaded module ntdll.dll at 00007FF820B00000
Debugger: Loaded module wow64.dll at 00007FF81E8E0000
Debugger: Loaded module wow64win.dll at 00007FF81EA70000
Debugger: Loaded module wow64cpu.dll at 0000000076EB0000
Debugger: Process entrypoint reached
Target method reached
Instrumented module flac.exe, code size: 438272
Debugger: Loaded module msvcrt.dll at 0000000076190000
Debugger: Unloaded module from 0000000076190000

flac 1.3.3
Copyright (C) 2000-2009  Josh Coalson, 2011-2016  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

Debugger: Exception c0000005 at address 00000000006C0518
Exception at address 00000000006C0518
Access address: 0000000000002420
Exception in instrumented module flac.exe 0000000000860000
Code before:
c7 0f 6a ff 0f fe f8 0f e2 fe
Code after:
0f 6e 0e 0f fe f9 0f 7e 3f 0f 73 f7 30 0f 73 d4
Process crashed
Debugger: Process exit
Found 4113 new offsets in flac.exe

[ifratric]

Hi, thanks for the detailed report. Could you first check if any of the following flags make a difference (I suggest trying them separately, not all together):

-generate_unwind
-patch_return_addresses
-stack_offset 1024

[De3mond]

Thanks for your response. I've tested these flags and found that using -patch_return_addresses can solve my current issue. I wander if i can get a brief explanation about this, in case i encounter similar problems in the future :)

[ifratric]

Please see https://github.com/googleprojectzero/TinyInst?tab=readme-ov-file#return-address-patching for more details.

Your case is unusual in the sense that, most of the time, when return addresses are an issue, it's because of C++ exceptions, but in that case -generate_unwind should also fix it. So I might take a look into what's going on, but probably won't get around to it until next week. If you can share the sample that triggers the issue (in\sample.flac), that would be helpful.

Are you using a 32- or 64- bit build of flac? You could try switching to the other build to potentially avoid -patch_return_addresses and its performance hit.

[De3mond]

attachment.zip

I am using a 32-bit build of flac. Additionally, this issue(run normal using DynamoRIO) also occurred when I fuzzed jp2klib.dll. I used these three flags, but the problem still be there. Detailed description follows. Now I have bundled the harness program, DLL file, and test cases for both flac and jp2k in the reply.

• prompt: litecov.exe -instrument_module JP2KLib.dll -target_module jp2kTest.exe -target_method main -nargs 2 -iterations 1 -persist -provided_flag -- jp2kTest.exe in\sample.jp2
• output:

Instrumented module JP2KLib.dll, code size: 622592
[i] JP2KCodeStm::Init() addr - 74309006
[i] JP2KCodeStm::read() addr - 74309161
[i] JP2KCodeStm::seek() addr - 7430918d
[i] JP2KCodeStm::IsSeekable() addr - 743090d4
[i] JP2KCodeStm::GetCurPos() addr - 74308fce
[i] MemObj initialized via JP2KLibInitEx()
img start[i] MemObj::alloc() - allocated 0x174 bytes at 15a1960
[i] MemObj::memset() - dest:0x15a1960   val:0x0   size:0x174
img finish[image addr] 15a1960
[i] MemObj::alloc() - allocated 0x58 bytes at 1598c20
[i] MemObj::memset() - dest:0x1598c20   val:0x0   size:0x58
[opt addr] 1598c20
[i] MemObj::alloc() - allocated 0x1f4 bytes at 15a1af0
[i] MemObj::memset() - dest:0x15a1af0   val:0x0   size:0x1f4
[initialized opt]
[structure at offset 0x14] 15a1af0
[i] MemObj::alloc() - allocated 0x40 bytes at 15a2048
[i] MemObj::memset() - dest:0x15a2048   val:0x0   size:0x40
Exception at address 00000000BAADF00D
Access address: 00000000BAADF00D
Process crashed
Found 687 new offsets in JP2KLib.dll

• without instrumentation output

[i] JP2KCodeStm::Init() addr - 74309006
[i] JP2KCodeStm::read() addr - 74309161
[i] JP2KCodeStm::seek() addr - 7430918d
[i] JP2KCodeStm::IsSeekable() addr - 743090d4
[i] JP2KCodeStm::GetCurPos() addr - 74308fce
[i] MemObj initialized via JP2KLibInitEx()
img start[i] MemObj::alloc() - allocated 0x174 bytes at 34960
[i] MemObj::memset() - dest:0x34960   val:0x0   size:0x174
img finish[image addr] 34960
[i] MemObj::alloc() - allocated 0x58 bytes at 2f058
[i] MemObj::memset() - dest:0x2f058   val:0x0   size:0x58
[opt addr] 2f058
[i] MemObj::alloc() - allocated 0x1f4 bytes at 34ae0
[i] MemObj::memset() - dest:0x34ae0   val:0x0   size:0x1f4
[initialized opt]
[structure at offset 0x14] 34ae0
[i] MemObj::alloc() - allocated 0x40 bytes at 24488
[i] MemObj::memset() - dest:0x24488   val:0x0   size:0x40
[i] JP2KCodeStm::Init()
[i] JP2KCodeStm::read() - writing 12 bytes to 0x55f9b0, bytes_written: 12
Memory @ 0x55f9b0, size: 12
00 00 00 0c 6a 50 20 20 0d 0a 87 0a
[i] MemObj::alloc() - allocated 0x10 bytes at 2d400
[i] MemObj::memset() - dest:0x2d400   val:0x0   size:0x10
[i] MemObj::alloc() - allocated 0x3c bytes at 2cb00
[i] MemObj::memset() - dest:0x2cb00   val:0x0   size:0x3c
[i] MemObj::alloc() - allocated 0x48 bytes at 2cb48
[i] MemObj::memset() - dest:0x2cb48   val:0x0   size:0x48
[i] MemObj::alloc() - allocated 0x19000 bytes at 34ff8
[i] JP2KCodeStm::read() - writing 102400 bytes to 0x34ff8, bytes_written: 14691
Memory @ 0x34ff8, size: 14691
00 00 00 0c 6a 50 20 20 0d 0a 87 0a 00 00 00 14
66 74 79 70 6a 70 32 20 00 00 00 00 6a 70 32 20
00 00 00 4f 6a 70 32 68 00 00 00 16 69 68 64 72
00 00 09 23 00 00 06 76 00 04 0f 07 00 00 00 00
00 0f 63 6f 6c 72 01 00 00 00 00 00 10 00 00 00
22 63 64 65 66 00 04 00 00 00 00 00 01 00 01 00
00 00 02 00 02 00 00 00 03 00 03 00 01 00 00 00
00 38 f4 6a 70 32 63 ff 4f ff 51 00 32 00 00 00
00 06 76 00 00 09 23 00 00 00 00 00 00 00 00 00
00 06 76 00 00 09 23 00 00 00 00 00 00 00 00 00
04 0f 01 01 0f 01 01 0f 01 01 0f 01 01 ff 52 00
0c 00 00 00 01 00 05 04 04 00 01 ff 5c 00 13 40
80 88 88 90 88 88 90 88 88 90 88 88 90 88 88 90
ff 64 00 25 00 01 43 72 65 61 74 65 64 20 62 79
20 4f 70 65 6e 4a 50 45 47 20 76 65 72 73 69 6f
6e 20 32 2e 34 2e 30 ff 90 00 0a 00 00 00 00 38
[i] MemObj::alloc() - allocated 0x3963 bytes at 51970
[i] MemObj::memset() - dest:0x51970   val:0x0   size:0x3963
[i] MemObj::memcpy_memset() - dest:0x51970   src:0x34ff8   size:0x3963
........
[i] MemObj::free_1() - 0x2d490
[i] MemObj::free_1() - 0x2d358
[i] MemObj::free_1() - 0x2d370
[i] MemObj::free_1() - 0x2cb00
[i] decoder returned: 0

[ifratric]

For the flac issue:

• Using builds from https://ftp.osuosl.org/pub/xiph/releases/flac/ (both 32 and 64-bit) I did not encounter any issues, though some 32-bit builds seem to have been built without DEP and thus require TinyInst flag -force_dep to be instrumented and produce coverage.
• I did find a build of 1.3.3. that exhibits issues. Looking into the code around the crash I see a function

.text:0043E736 sub_43E736      proc near               ; CODE XREF: sub_43E6A0+A5↓p
.text:0043E736                 mov     eax, [esp+0]
.text:0043E739                 retn
.text:0043E739 sub_43E736      endp

That is reading the return address from the stack and doing stuff with it, so in this case using -patch_return_addresses does seem like the correct fix (even though, for performance reasons, I'd suggest using a flac build where -patch_return_addresses is not needed).

For jp2klib.dll, I haven't looked into that yet, but note that I can't run your binaries. Can you provide me with a source code for your harness?

[ifratric]

And I believe I know what the issue with jp2klib.dll is: The harness uses Microsoft Detours, which is another instrumentation/hooking library. TinyInst does not support running alongside other instrumentation/hooking libraries, however TinyInst has its own Hook API (see https://github.com/googleprojectzero/TinyInst/blob/master/hook.md) that you could potentially use instead, which will avoid incompatibilities.

Specifically, in this case:

• If TinyInst instruments the module before Detours, then detour instruments code that's no longer used so basically you get code without Detours. That's what's happening in the example you provided.
• If Detours instruments code before TinyInst (e.g. by specifying a target_method that gets executed after Detour instrumentation), then TinyInst sees code (from Detours) it doesn't know how to handle (specifically in this case, relative jumps to code outside the current module). Is this is the only problem however, this might be fixable. I'm looking into this.

[ifratric]

With the latest TinyInst commit and using this harness from https://github.com/ronwai/jp2k_fuzz it seems to work, at least on a simple example, but only if target_method gets called after all the DetourAttach/DetourTransactionCommit calls as explained above. Example command line:

litecov.exe -instrument_module JP2KLib.dll -trace_debug_events=0 -trace_basic_blocks=0 -target_module jp2harness.exe -target_method fuzz_jp2k -nargs 1 -iterations 1 -persist -loop -- jp2harness\Release\jp2harness.exe sample.jp2

[De3mond]

For the jp2klib.dll issue, i believe the reason causing the issue is Detours. But I rebuilt the harness(also from https://github.com/ronwai/jp2k_fuzz) and winafl(with latest TinyInst commit), both using 32 bit build. In this case, i still encountered the crash issue as follow.

• prompt: litecov.exe -instrument_module JP2KLib.dll -trace_debug_events=0 -trace_basic_blocks=0 -target_module jp2kTest.exe -target_method fuzz_jp2k -nargs 2 -iterations 1 -persist -loop -- jp2kTest.exe in\sample.jp2
• output:

[i] JP2KCodeStm::Init() addr - 50079006
[i] JP2KCodeStm::read() addr - 50079161
[i] JP2KCodeStm::seek() addr - 5007918d
[i] JP2KCodeStm::IsSeekable() addr - 500790d4
[i] JP2KCodeStm::GetCurPos() addr - 50078fce
[i] MemObj initialized via JP2KLibInitEx()
Instrumented module JP2KLib.dll, code size: 622592
[!] WARNING: Relative jump to a differen module in bb at 5007918D

[!] WARNING: Relative jump to a differen module in bb at 50079161

[!] WARNING: Relative jump to a differen module in bb at 500790D4

[i] MemObj::alloc() - allocated 0x174 bytes at 4c890
[i] MemObj::memset() - dest:0x4c890   val:0x0   size:0x174
[image addr] 4c890
[i] MemObj::alloc() - allocated 0x58 bytes at 4a000
[i] MemObj::memset() - dest:0x4a000   val:0x0   size:0x58
[opt addr] 4a000
[i] MemObj::alloc() - allocated 0x1f4 bytes at 4ca20
[i] MemObj::memset() - dest:0x4ca20   val:0x0   size:0x1f4
[initialized opt]
[structure at offset 0x14] 4ca20
[!] WARNING: Relative jump to a differen module in bb at 50079006

[!] WARNING: Relative jump to a differen module in bb at 50078FCE

[i] MemObj::alloc() - allocated 0x40 bytes at 4cf78
[i] MemObj::memset() - dest:0x4cf78   val:0x0   size:0x40
[!] WARNING: Attempting to execute an instruction TinyInst couldn't translate
[!] WARNING: This could be either due to a bug in the target or the bug/incompatibility in TinyInst
[!] WARNING: The target will crash now
Exception at address 4FDD9D4A
Access address: 00000000
Exception in instrumented module JP2KLib.dll 50050000
Code before:
00 00 c6 05 14 01 d3 4f 01 cc
Code after:
c6 05 00 00 00 00 00 c6 05 15 01 d3 4f 01 85 f6
Process crashed
Found 30 new offsets in JP2KLib.dll

[ifratric]

Probably TinyInst version that WinAFL uses needs to be updated. The version you are running does not include the patch from yesterday (the latest version doesn't print the "Relative jump to a differen module in bb" warnings, see 2d9472e

[ifratric]

WinAFL just got updated with the latest TinyInst