Need to Rollback the data if the no of rejection exceeds 50

[VenkiSagaMicroService]

Hi Dimitri,

I load CSV data into postgres database using pgloader. Need to rollback the inserted data if no of rejected record count exceeds 50 . Could you please advise how to achieve this.

Regards,
Venkatesh S

[dimitri]

Today pgloader counts rejected rows and writes them to a .rej file, but it
always finishes the load regardless of how many rows were rejected. There is
no way to say "if more than N rows are bad, something is wrong with the source
file — stop early."

Design

Two new WITH options that can be used independently or together:

LOAD CSV FROM 'data.csv' ...
WITH max errors 50,          -- abort after 50 rejected rows (absolute)
     max error ratio 0.01;   -- or after 1 % of rows are rejected

The rejection counter is maintained in the batch writer alongside the existing
stats. When either threshold is crossed the writer signals an abort: the
in-progress COPY batch is rolled back and pgloader exits with a non-zero
status and a clear message stating how many rows were rejected and which
threshold was hit.

Important caveat: pgloader loads data in batches of ~1 000 rows, each
committed independently. There is no single wrapping transaction around the
entire load. When the threshold is crossed, already-committed batches are
not rolled back — pgloader aborts forward progress but leaves the target
table in a partially-loaded state. You get to keep the pieces. If an
all-or-nothing guarantee is required, wrap the load in a pre/post SQL step
that truncates the table on failure, or use a staging table and swap on
success.

---

If you'd like to see this feature prioritised, sponsoring its development is
possible through The Art of PostgreSQL OSS sponsorship
program. Sponsored features get
dedicated implementation time and are tracked to completion; the design above
is already in place and ready to go.