Implement Results for Amended Sample Workflow

[kujo4pm]

Steps to reproduce

• Create a new Lab request in Tamanu
• Sync to Senaite
• Add results to new Sample
• Publish Sample
• Sync to Tamanu
• Invalidate the sample in Senaite - this will invalidate the old sample and needs to create a new duplicate sample
• Add results to new sample
• Republish
• Check the result have been updated in Tamanu

Current behaviour

The PDF is recreated with an invalidated value but the new raw data is not sent across.

Expected behaviour

We need the updated analysis result data of the newly created sample to be sent across (as Observations) - the diagnostic report associated with the original ServiceRequest.
For this, I think we should implement: DiagnosticReport.status = amended for any Sample that is invalidated and a new one created. See here: https://hl7.org/fhir/R4B/valueset-diagnostic-report-status.html

For any DiagnosticReport received with an amended status, Tamanu should check the dateTime and ensure that it is updated in Tamanu only if the modified datetime is updated is newer than that originally received. Theoretically samples could be invalidated an unlimited number of times and as as there is no garantee the tasks are handled in order Tamanu may receive an older result after a newer one.

[rohan-bes]

Ah I see the issue @kujo4pm, there is a bit of complexity here hey.

Do you know if it would be possible for SENAITE to order the DiagnosticReports it sends to Tamanu by their lastUpdated (per request ID)?
ie.

If SENAITE has a 3 DiagnosticReports for RequestID XYZ:

{ 
  id: A, 
  basedOn: XYZ, 
  status: preliminary, 
  lastUpdated: 12:00 
},
{ 
  id: B, 
  basedOn: XYZ, 
  status: ammended, 
  lastUpdated: 12:01 
},
{ 
  id: C,
  basedOn: XYZ, 
  status: final, 
  lastUpdated: 12:02 
}

Then it would ensure it sends A before B and B before C. And if it fails to send A, it doesn't send B, etc.

I appreciate this is a bit more work for you, but it would ensure that SENAITE and Tamanu have a consistent version history. If we went with the "Tamanu only accepts new DiagnosticReports with a greater lastUpdated than it's previously received" approach, then if C happened to be sent first, users in Tamanu wouldn't know the full history of the request.

Possibly that history isn't of interest to users in Tamanu, but I imagine there could be some possible state transitions that might seem odd without history (eg. if we received A, followed by C, then it might be surprising that the Published results don't match the interim results).

If what I'm asking for is too hard, I think we can definitely implement the approach you suggested. There's also potential to use resource versioning here if we like to avoid any time based shenanigans? But if you're already have lastUpdated that might be easier

[kujo4pm]

Good point. I forgot that Tamanu keeps history. The root issue is that it's hard to retrospectively update history right? This is probably a good design decision!

One quick note in case it changes anything. The amended comes after final. But again no guarantee on order:

{ 
  id: A, 
  basedOn: XYZ, 
  status: preliminary, 
  lastUpdated: 12:00 
},
{ 
  id: B, 
  basedOn: XYZ, 
  status: final, 
  lastUpdated: 12:01 
},
{ 
  id: C,
  basedOn: XYZ,   
  status: amended, 
  lastUpdated: 12:02 
}

Will discuss this with Jordi. Hang tight.

[kujo4pm]

@rohan-bes I spoke with @xispa about this and I had my wires crossed. Apologies for that. The unpredictable synchronicity issue I mentioned (since crossed out in the issue description) was regarding how different subscribers will not know which order they will receive a notification. This is not an issue and is not relevant to this. We can be sure that Tamanu will receive events in the order in which they occur.

Instead this resolves to the following Workflow:

Workflow description

• A ServiceRequest is fetched. In Senaite a sample is created linked to that servicerequest via id.
• Test results are added, verified and finally published.
• On publish a DiagnosticReport is sent to Tamanu (with status final).
  Then it is discovered that the published results were wrong.
• The sample is duplicated and the old is set to invalidated. The new is then linked to the initial ServiceRequest.
• A new DiagnosticReport (with status entered-in-error) is sent to Tamanu - this time with no Observations but a PDF saying the results are invalidated.
• **New Action 1: ** In Tamanu all the results should be cleared or signalled as invalid somehow. They are no longer valid.
• **New Action 2: ** The new Sample is processed and the DiagnosticReport should be sent through as normal but this time should be sent through as amended.

So now this card resolves to two new pieces of work:

1. When Tamanu receives a DiagnosticReport with status = entered-in-error - all results should be invalidated. Whether they are cleared or signalled to be not valid is of course your call.
2. Senaite needs to send future DiagnosticReports after an invalidated with status amended.

[rohan-bes]

Hi @kujo4pm, sorry for the late reply. Yep this sounds good, I'll update Tamanu so that:

1. If the Lab Request is 'Published' in Tamanu and we receive status=entered-in-error, we transition to 'Invalidated'
2. If the Lab Request is 'Invalidated' and we receive status=amended, we transition to 'Published' (there's no 'Amended' status in Tamanu atm)

Out of curiosity would SENAITE send any subsequent amended DiagnosticReports? Or would SENAITE always send an entered-in-error DiagnosticReport between multiple amends?

[xispa]

Out of curiosity would SENAITE send any subsequent amended DiagnosticReports? Or would SENAITE always send an entered-in-error DiagnosticReport between multiple amends?

• If the lab re-publishes the results, you will receive an amended DiagnosticReport without an entered-in-error in between. The likelihood of this scenario is very low, because once a sample and its analyses have been published, they can no longer be modified. Therefore, re-publishing would generally only make sense if the lab wants to generate a results report using a different template or similar reporting settings.

• If the lab invalidates the sample, you will receive a DiagnosticReport with status entered-in-error. An amended report will follow afterwards (perhaps 1 hour or 2 days later), once the retest sample (a copy of the invalidated one) has been published. Note that this process can occur indefinitely:

  publish → invalidate → [retest] → publish → invalidate → [retest] → ...

  In this scenario, you will always receive an entered-in-error between multiple amends

[kujo4pm]

Ok so on the Senaite side - this issue resolves to Senaite will update any publishes after a publish as status amended. This will handle the two cases mentioned just above.

@xispa please confirm.

[xispa]

@xispa please confirm.

Yes. Note that amended status does not exist in SENAITE though. SENAITE will be pushing DiagnosticReport in partial (if not yet published), final (if published) or entered-on-error (if invalidated) statuses. See SAMPLE_STATUSES mapping.

Events involving the invalidation of a sample

1. verify sample → POST DiagnosticReport (partial), without PDF
2. publish sample → POST DiagnosticReport (final)
3. invalidate sample → POST DiagnosticReport (entered-on-error)
4. verify retest-1 sample → POST DiagnosticReport (partial)
5. ... n(publish+invalidate+verify) ...
6. publish retest-n- sample → POST DiagnosticReport (final)