Description:
The transaction extraction prompt currently relies on hardcoded, specific merchant names (e.g., URBAN COMPANY LIMITEDGURUGRAM) to teach the LLM how to parse HDFC reward point reversals. Testing reveals that if these specific examples are replaced with generic placeholders, the LLM fails to identify the -[Points] + [Amount] pattern as a refund, instead extracting it as a debit->purchase.
Reproduction Steps:
- Locate the few-shot examples in
CC_TRANSACTIONS_PROMPT.
- Replace specific merchant names with generic placeholders (e.g.,
Generic Merchant).
- Parse a statement containing a refund formatted as
-Points + Amount.
- Observed: The LLM extracts the transaction as
type: "debit".
- Expected: The LLM should extract
type: "credit".
Impact:
- Verification Failure: Misclassifying a refund as a debit breaks the CC reconciliation formula.
- Fragility: Refunds from merchants not named in the prompt are likely misclassified.
Proposed Solution:
None at this time
Description:
The transaction extraction prompt currently relies on hardcoded, specific merchant names (e.g.,
URBAN COMPANY LIMITEDGURUGRAM) to teach the LLM how to parse HDFC reward point reversals. Testing reveals that if these specific examples are replaced with generic placeholders, the LLM fails to identify the-[Points] + [Amount]pattern as a refund, instead extracting it as adebit->purchase.Reproduction Steps:
CC_TRANSACTIONS_PROMPT.Generic Merchant).-Points + Amount.type: "debit".type: "credit".Impact:
Proposed Solution:
None at this time