Ordering of HybridGibbs

[jeverink]

Description

The order in which the HybridGibbs samples from the conditional distributions is currently the order in which they are combined in the JointDistribution. This could cause confusion when providing user-specified initial-guesses for some variables, as these initial-guesses might not be used at all, impacting the results.

It would perhaps be more natural to use the ordering in which the sampling strategy is specified instead, such the ordering is explicitly described as part of the sampler, not as part of the distribution.

Definition of Done (Feature/change)

• Add scan_order argument for HybridGibbs sampler to provide user defined order (if none is provided sampling_strategy order is followed)

Definition of Done (Mandatory)

• Documentation added (docstrings on all public methods/classes)
• Unit tests added/updated (and passing!)
• Code reviewed and approved by 2 CUQI-maintainers
• Online documentation is rendered correctly (esp. math)

Defintion of Done (Optional)

• (skip for this one) HowTo/tutorial added

Example to reproduce (behavior before this PR)
With this PR, sampling order is specified with scan_order or sampling_strategy if the former is None

Specific problematic code from the HybridGibbs class:

[amal-ghamdi]

Thank you @jeverink for raising this issue and implementing a solution to control the order. This is very useful.

I wanted to ask you if you have considered adding a parameter to HybridGibbs to specify the ordering directly. e.g. HybridGibbs(..., sampling_order=['x', 'd'], ...), instead of relying on the strategy. I can see how the latter approach is more compact, but the first approach is more explicit and prevent user unintended errors. (both inferring the order from the posterior and from the sampling strategy is implicit )

[jeverink]

Thanks for the feedback @amal-ghamdi ,
Using a sampling_order parameter is actually how I initially implemented it in the code for my thesis.
There is also the option to make the scan order random, which is more efficient for some models, and was considered as part of the long dormant #196.

I will add this functionality shortly.

[amal-ghamdi]

@jakobsj , DoD updated for your review. Thank you.

[jakobsj]

Nicely described issue with clear example, DoD approved @amal-ghamdi