Enabling the use of MS-DIAL5 with Rapid-QC-MS

[kozo2]

Proposed mentors: @kozo2 @htsugawa

Languages/skills: Python (Dash), Understanding the Rapid-QC-MS system design and controling MS-DIAL5 (console app) for it.

Estimated Project Length: 175 hours

Difficulty: medium

Rapid QC-MS is an all-in-one solution for automated quality control of liquid chromatography-mass spectrometry (LC-MS) instrument runs, both during and after data acquisition.
It offers a fast, straightforward approach to ensure collection of high-quality data, allowing for less time investigating raw data and more time conducting experiments.

Rapid QC-MS is dependent on MS-DIAL4 (console app) and currently does not have compatibility with MS-DIAL5.
Therefore, in this idea, GSoC contributor will work to ensure compatibility and enable Rapid QC-MS to function with MS-DIAL5 as well.

[nparikhqmc]

Transactional Replication — Operational Q&A

Following up on the replication channel options above with answers to some operational questions.

Performance impact on V2 primary

Minimal under normal conditions. The Log Reader Agent runs continuously, adding small sustained I/O and CPU overhead. The larger risk is log truncation: replication holds the TLOG open until the Log Reader processes those records. If the agent falls behind (VPN blip, outage), the TLOG cannot be truncated and will grow until the agent catches up. For Option B (native transactional replication), backlog lands in the distribution database (bounded by retention, default ~72 h) rather than directly in the TLOG — so the primary is somewhat protected, but a sustained outage beyond retention forces a re-seed.

Compatibility with regular backups

Replication and backups are fully independent — they coexist without conflict. Regular full, differential, and log backups continue unchanged. The only interaction is that replication holds the TLOG until the Log Reader processes it, so log backups may be slightly larger, but the backup jobs themselves are unaffected.

Backups also serve as the re-seed mechanism: if a subscription needs to be reinitialized after a long outage, restoring a backup to the shadow and initializing from that LSN avoids running a fresh snapshot agent against the 4 TB live DB.

Recovery model note: Option B does not require V2 to be in full recovery model. If point-in-time restore capability is also wanted from backups, full recovery is needed — but that is a backup requirement, not a replication one.

How log truncation works

Truncation requires two gates to both clear:

1. A log backup has been taken
2. The Log Reader Agent has processed past that point

Both must be satisfied. Even in simple recovery model (where truncation normally happens at checkpoints), the Log Reader gate still applies — the log can grow if the agent falls behind. If the TLOG grows unexpectedly, check whether the Log Reader Agent is running and caught up, not just whether log backups are firing.

Initializing the shadow from a full backup

The correct initialization sequence — do not stop backups during this window:

1. Keep log backups running normally on V2
2. Take a full backup (note the LSN)
3. Restore the full backup to the shadow
4. Apply each subsequent log backup to the shadow in sequence to close the gap to real-time
5. Initialize the replication subscription from that LSN
6. Replication takes over continuously from that point forward

Stopping log backups during steps 2–5 would break the log chain and make it impossible to apply logs sequentially to bring the shadow current. The log backups serve double duty during this window: normal V2 safety net and the catch-up mechanism for the shadow.

The one thing to watch: do not let truncation clear logs the shadow still needs. Since the Log Reader Agent is not running yet during initialization, replication's truncation gate is not in place — keep applying logs to the shadow faster than they are being truncated on V2.

[kozo2]

I am closing this issue because I have left MS-DIAL development team.