Description
In WarpParse V0.23.4, runtime performance drops significantly after enabling PostgreSQL knowledge base enrichment logic.
The input data used in this test is a single repeated record. In theory, the PostgreSQL query result should be cached after the first lookup.
However, the observed EPS still drops significantly, which suggests that the PostgreSQL query cache may not be working as expected.
Evidence
Based on the runtime statistics:
| Scenario |
EPS |
| Without PostgreSQL knowledge base lookup |
~170,000 |
| With PostgreSQL knowledge base lookup |
~86,000 |
Because the input data is a single repeated record, this scenario should be highly cache-friendly.
Expected Result
When the input data is highly repetitive, PostgreSQL knowledge base lookup should hit the cache after the first query.
If the cache works correctly, EPS should not drop from around 170k to around 86k, which is close to a 50% performance degradation.
Actual Result
After enabling PostgreSQL knowledge base lookup, EPS drops from around 170,000 to around 86,000.
Version
- WarpParse: V0.23.4
- Related feature: PostgreSQL knowledge base enrichment / lookup logic
Description
In WarpParse V0.23.4, runtime performance drops significantly after enabling PostgreSQL knowledge base enrichment logic.
The input data used in this test is a single repeated record. In theory, the PostgreSQL query result should be cached after the first lookup.
However, the observed EPS still drops significantly, which suggests that the PostgreSQL query cache may not be working as expected.
Evidence
Based on the runtime statistics:
Because the input data is a single repeated record, this scenario should be highly cache-friendly.
Expected Result
When the input data is highly repetitive, PostgreSQL knowledge base lookup should hit the cache after the first query.
If the cache works correctly, EPS should not drop from around 170k to around 86k, which is close to a 50% performance degradation.
Actual Result
After enabling PostgreSQL knowledge base lookup, EPS drops from around 170,000 to around 86,000.
Version