The transaction processing flow should be restructured into a more flexible and less error prone.
Transactions can be retrieved from Nordigen in batches and processed parallel to that. This alone will use less memory than before.
Besides that we should have a factory which defines a flow to be able to easily turn flow features on and of and maybe make it modifiable by the user. Also we have to remove importing the requisitions from this flow. That should be seperate and happen before the transaction import. We also import the accounts and not the requisitions itself.
Asset account import flow:
- Retrieve requisition
- Retrieve accounts
- Compare accounts with Firefly III asset account
- Import the new ones
- Store accounts in DB
- New accounts should be marked as new / have a recalculate balances flag
Base flow:
- Retrieve paginated transactions for period from Nordigen
- Retrieve transactions within same range of the current page from Firefly III
- Store a list of hashes (or normalized strings) of the Firefly III transactions for checking against
- Convert transaction from API model to an object we understand
- The next steps should be a per transaction flow:
- Check for existing transactions against the existing hashes list
- Check for duplicates in current import
- Maintain a list of hashes (or normalized strings) that we already have imported.
- Whenever we hit a duplicate check if we can extend the currently imported transaction (and do that)
- Check whether the account is one of the asset accounts
- Check whether it could be a transfer from an internal savings account
- Map it to the correct account
- Post it to Firefly III
- Trigger an transaction processed event
- That event should cause a calculation of balances for accounts marked as new or having a recalculate balances flag
A few requirements
- Code should be concise and readable (ie. a factory that defines the import flow and another that defines the transaction processing flow)
- Batches that have been imported should not be references any more and not be held in memory
- Notifications should be send whenever it is not possible to process a transaction, with clear details about the transaction
- Logs should be generated for all the steps. For diagnostics purposes and for showing in the import log
The transaction processing flow should be restructured into a more flexible and less error prone.
Transactions can be retrieved from Nordigen in batches and processed parallel to that. This alone will use less memory than before.
Besides that we should have a factory which defines a flow to be able to easily turn flow features on and of and maybe make it modifiable by the user. Also we have to remove importing the requisitions from this flow. That should be seperate and happen before the transaction import. We also import the accounts and not the requisitions itself.
Asset account import flow:
Base flow:
A few requirements