ADR to support full automation of dss publishing#26
Conversation
Why are these changes being introduced: * full automation is required to support this service longterm How does this address that need: * documents a proposed change to allow full automation
JPrevost
requested review from
ehanson8,
hakbailey,
jazairi,
matt-bernhardt and
osmandin
jazairi
left a comment
jazairi
left a comment
There was a problem hiding this comment.
This seems solid to me from the ETD perspective. 👍
|
|
||
| We will allow an optional Callback URL in the input message to DSS. | ||
|
|
||
| Upon finishing processing incoming messages, DSS will call all unique callback URLs it encountered. |
There was a problem hiding this comment.
Is there a scenario where two upstream applications posting an overlapping batch of messages could cause a problem here? Let's say:
- ETD and Wiley both start sending messages to the queue at roughly the same time.
- Wiley finishes sending messages first and starts DSS processing. ETD is still sending messages in its batch.
- DSS starts processing messages. Its callback list contains both the Wiley and ETD callback URLs.
- For whatever reasons around lag time or just a job that takes a while, DSS gets through all the messages and ETD is still sending its batch, but there's a longer than 20 second lag between messages sent to the queue from ETD.
- DSS gets an empty response after long polling and calls the Wiley and ETD callback URLs.
- ETD keeps sending messages. When it finishes the batch, it starts DSS again.
- DSS processes the remaining ETD messages from the batch and calls the callback URL for those. This may happen while the first post-callback process is still happening in ETD (another overlapping process scenario).
I think this is both super unlikely and will be fine if it happens, but it does mean the post-callback output process used by the upstream apps should be able to handle multiple callback jobs for a single batch and ensure those jobs don't conflict with each other, just in case.
Am I missing anything? I know this is super unlikely but I don't want us to end up with borked data in ETD because of some bizarre outlier scenario.
There was a problem hiding this comment.
I think you are both correct that this is unlikely and still needs to be accounted for. ETD/Wiley would need to be able to know what to do if they get a message to start the output queue processing. They might choose to check if they are already processing the queue (and thus ignore the second message) or send an alert that something weird happened or... something else.
We'll similarly want to ensure the DSS fargate task checks to see if it is already running before spinning up an additional task to avoid sending more data to dspace than it can handle.
matt-bernhardt
left a comment
matt-bernhardt
left a comment
There was a problem hiding this comment.
As perhaps the farthest removed of all reviewers from this application, I'm not sure my review should carry much weight - but this seems like a reasonable approach based on what I do know.
From what Helen and Jeremy as describing, it seems like the upstream applications are where this scenario would need to be anticipated?
5 participants
NOTE: this is a proposal. Feedback is essential to ensure we understand the needs of both the ETD and Wiley solutions. I believe this makes sense, but am open to changes to better support needs I haven't thought of.
Why are these changes being introduced:
How does this address that need:
Includes new or updated dependencies?
NO
Changes expectations for external applications?
YES