Core Library Overview

[bHimes]

Summary

In general, cisTEM relies too heavily on homespun code. Many routines would likely perform better using optimized numeric libraries. There is also a huge amount of copy pasta and oddly named code. I have personally written several methods to (sometimes years) later discover there was an existing function that just did jump out at me. Aside from clear naming (which will never be clear to everyone) breaking up compilation units and grouping them by function would also help.

Notes

• The Image class is a target, but should just be built from the ground up. It is too widely used to replace piecemeal and too flawed to fix.

• File i/o, generally okay, but there are a few major inefficiencies

• an mrc read reads everything in, then reorders in memory after the read, rather than just reading to the correct layout

• i/o classes explicitly assume we want a float (4 byte) which is often not the case.

• the directory layout can be compartmentalized, which will make it easier to group files by function and also then to include CLAUDE.md specific to that functionality in those folders.

• Several older formats/ conventions remain supported that are probably okay to cut loose. For example frealign parameter files

• Similar to the image class, we have our own homespun star file i/o where we could probably just use gemmi

• We use the binary star files which is an attempt to fix a performance issue with text files and to maintain independence of the DB for recon/refinment progs. I think we should get rid of this (after timing to ensure it is okay) and instead move the refinment params to a SECOND DB.

  • you can attach to multiple DBs, which we can handle safely.
  • We would still need a db -> starfile program (either with cisTEM star file, or gemmi) but that still cuts out the additional binary starfile.

• optimizers, we have a very limited and archaic set of optimization tools.

  • newer and more robust options should be considered
  • the framework to run on variable optimizers should be considered

• electron dose should have an option to populate a curve object rather than a full 2d. It was written assuming it would only be used for unblur

• utils should be separated out (stopwatch, progress bar, user input, file utils etc.)