Use cases

[rhfogh]

This discussion should collect use cases and discuss how we want to accommodate them. Here are some to start with:

1. Single crystal in holder, single sweep:

• 1 (mesh scan + line scans) (or manual centring)
• 1 characterisation images (or sweep) (optional)
• 1 raw diffraction dataset (single HDF5/mini-cbf dataset)
• 1 MTZ file

NOTE: You can have more than one experiment per sample
NOTE: Characterisation and mesh/line scans are not really relevant once you start processing and should ideally be hidden in views.
NOTE: Characterisation images may be a sweep, just like the data, but should not be processed with the data.

2. Single crystal in holder, 1 wavelength, N (orientation sweep), no interleaving:

OR Multiple crystals in 1 holder, 1 wavelength, N (crystal, sweep), no interleaving (as suggested by Anamga)

• 1 characterisation images (or sweep)
• N (mesh scan + line scan) (or manual centring)
• N raw diffraction datasets (HDF5/mini-cbf datasets)
• 1 resulting merged reflection data (MTZ file)

NOTE: To view or list multi-sweep results you must have a line per processing calculation, not a line per dataset or per MTZ file
NOTE the multi-crystal case could also be handled under use case 4.
NOTE You would want a single Sample record for all the crystals, describing contents, expected symmetry, etc.

3. Single crystal in holder, 1 orientation, N (wavelength sweep), interleaving in M wedges:

• 1 (mesh scan + line scan) (or manual centring)
• 1 characterisation images (or sweep) (optional)
• N*M raw diffraction datasets (HDF5/mini-cbf datasets) # potentially ... depends on detector arming, HDF5-vs-mini-cbf etc
• N resulting merged reflection data (MTZ files)

4. N crystals in one or more holders, one orientation per crystal

• 0 or N characterisation images
• ?? (mesh scan , line scan)
• N raw diffraction datasets (HDF5/mini-cbf datasets)
• 1 resulting merged reflection data (MTZ file)

NOTE: This could be images or small wedges, XFEL or SSX, plate, stream or grid.
NOTE: For multiple crystals in one holder, large enough to support multidegree sweeps, you could also use Case 2 (Anamga).
NOTE: The natural organisation would be one Sample record per grid/vial, (and not per crystal)
NOTE: For these experiments you might need to filter the data sets to make one (or more) combinations of image sets for processing. The sets could come from multiple Samples and experiments, and you might need to reprocess the image selection. Summa Summarum: you need to be able to specify a Dataset that is a group or collection of other Datasets.

5. (Re)processing input parameters

We would want to track which Datasets are used as input to which calculations, and how. First that requires connecting input Datasets, job parameters, results and output Datasets, With some processing programs that might also require parameters that are specific not to the Dataset, nor to the Program run, but to the combination. Examples:

• Distinguishing between input reference MTZ files and data MTZ files.
• (Re)processing using only a subset of the images in a Dataset.

NOTE: These parameters cannot be put in the dataset, since they only apply to a single program run. There are basically two alternatives:

• Make a copy of the Dataset where necessary, to have a place to put the extra parameters.
• Have an additional ‘copy of’ object that points to both the processing job and the data set without copying the Dataset contents.

6. Complex interleaved sweeps

Under workflow control it is possible to do simultaneous interleaving of several parameters, e.g. combined wavelength interleaving and inverse beam. The resulting patterns could be quite complex. It is for instance not given that all wedges have the same width - the strategy might well combine successive wedges to reduce the number of omega changes. Representing this situation would require either a complex model for specifying interleaving strategies, or specifying the series of wedges (scans) and their acquisition order explicitly.

7. Broken-up sweeps

GPhL characterisation strategies currently uses a series of five separate 1.2 wedges taken from a single 181.2 sweep. The resulting images are numbered 1-12, 451-462, 901-912, 1351-1362 and 1801-1812, and are processed by XDS as a single sweep., with holes. Some future GPhL phasing strategies might also include chopping a single continuous sweep up into pieces and acquiring them out of order.

8. Single crystal holder – pin

A mounted pin contains a loop with (most often) a single crystal, with known origin and composition. The Unit cell and crystal form might be know, partially known, or one of several known alternatives.

9. Multi-crystal holder – drop.

A drop could contain several crystals with the same origin, composition, and unit cell and crystal form information. Crystals could be identified by mesh scan, or have their locations pre-identified.

10. Multiple holder – plate

A plate contains multiple wells, each with its own defined location, sample origin and composition. The same would hold for multi-location pins, pucks, or Dewars.

11. Formatted multi-crystal holder – chip

A chip contains a very large number of predefined locations, each potentially with its own crystal. Information about sample origin, composition, and unit cell and crystal form attaches to the vial used to ‘populate’ one or several entire chips.

12. Unformatted multi-crystal holder

This could be a stream or flattened cubic lipid phase. These may contain a very large number of crystals that must be identified by scanning or appearance time. Information about sample origin, composition, and unit cell and crystal form attaches to the vial used to ‘populate’ one or several entire chips.

[anamga]

Thanks for starting this discussion and the good summary of the use cases
Some comments:
For use cases 1-3 the mesh scan would generally be done before characterization - there may be 0 characterization images (if the user knows how they want to collect data or if images in the mesh scan are used for characterization) or 0 mesh scans (if crystal centering is done by hand).
Case 4 is complex, because in some cases (several distinct large crystals on a small grid/loop) the wedges could be a complete or fairly complete data set. Even in a crystallization plate it might be possible to collect 30 degrees of data or so from individual crystals - for processing purpose this would be similar to use case 2, although with data from different crystals instead of different sweeps. For true SSX (still or very limited rotation per crystal, or flow based sample delivery) handling is quite a bit different, so maybe it should be in a different use case.
Processing requires some thought. In general, 'copy of object' sound better, but probably there are pros and cons to both options. I would prefer not to copy original input images, unless the format is changed for convenience (we do that at MAX IV with interleaved data sets, we "repack" the data sets collected at the same wavelength into a single data set)....there would always be someone who would attempt to sftp the whole processing tree...

[rhfogh]

Very good and useful points. I have updated the use cases to show that the case of several largish crystals on a single loop could be handled either under use case 2 or use case 4, and to incorporate your comments about mesh scans and characterisation.