duckdb: reuse DatabaseInstance across iterations to fix harness throughput#74
Open
tc-agent wants to merge 2 commits into
Open
duckdb: reuse DatabaseInstance across iterations to fix harness throughput#74tc-agent wants to merge 2 commits into
tc-agent wants to merge 2 commits into
Conversation
Fuzzing Coverage ReportTested: project OSS-Fuzz: Fuzz Introspector · build status
Per-harness
Δ = (after − before) / before, to accommodate that denominators may change. "new" when before is 0; "deleted" when after is 0. |
tc-agent
force-pushed
the
update-duckdb-fuzzing
branch
from
May 28, 2026 17:06
8babf8b to
e35ce3a
Compare
tc-agent
force-pushed
the
update-duckdb-fuzzing
branch
from
May 28, 2026 18:05
e35ce3a to
f56bf25
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make the duckdb
parse_fuzz_testharness usable for fuzzing again. This is a throughput / build fix, not a coverage push.Problem
Since DuckDB landed its PEG-based parser,
ParserCachehas lived on theDatabaseInstance. The upstream harness constructs a freshduckdb::DuckDB(nullptr)per iteration, which rebuilds the matcher tables from scratch on the first query of each fresh instance. Under ASan that rebuild costs tens of seconds. When a fuzz input hits theSimilarEntry→CreateDefaultEntriespath it can exceed the libFuzzer-timeout=25on the very first iteration; even when it doesn't, throughput stays near 0 exec/s.Production runtime line coverage is 11.26% per Fuzz Introspector (48,402 / 429,990 — also visible in the coverage report), which is consistent with the fuzzer doing only a handful of executions before its timer expires.
What changes
Reuse one
DatabaseInstanceacross iterations and prime the PEG matcher inLLVMFuzzerInitialize.LLVMFuzzerInitializeis not bounded by the per-input timeout, so the one-off matcher build no longer races the 25 s budget. Per-input cost drops from seconds to milliseconds; the harness reaches ~9 exec/s under ASan on a 4-core build (vs. effectively 0 today).The
DatabaseInstanceis shared across iterations, so catalog state (CREATE TABLE / CREATE INDEX / etc. that succeed) accumulates over the run. Each iteration uses a freshConnection; theDatabaseInstanceand any persisted catalog entries outlive it. This is called out in a comment ong_db.Add
-I./third_party/fmt/includeto the harness compile socast_helpers.hpp's#include "fmt/format.h"resolves. This matches theinclude_directories(third_party/fmt/include)already inCMakeLists.txtused for the in-tree CMake build oflibduckdb_static.a; only the standalone harness compile was missing it. The last successful production build is recorded at OSS-Fuzz (last_successful_build.finish_time= 2026-05-28T07:28:50Z); the include is preventive against the same surface breaking in a future build.Note on the harness file
projects/duckdb/harnesses/parse_fuzz_test.cppshadows the upstream copy attest/ossfuzz/parse_fuzz_test.cpp;build.shoverlays it onto the source tree before compiling so the matcher-reuse change is applied without waiting for an upstream round-trip. The same fix is tracked upstream at tc-agent/duckdb#1; the overlay can be retired frombuild.shonce that lands.This PR intentionally does not change harness coverage scope. Follow-up work to add CSV/JSON and Parquet harnesses + seed corpus generation will go in a separate PR.