transforming star file

[pherepanov]

Dear Daniel,

I am trying to transform a star file (from a single particle project, Relion 5.0) to superpose subparticles based on Chimera transformation matrix. I used both star.py and subparticles.py, which produce different results but neither allow reconstruction of correctly oriented and positioned particle. Here are two commands I used:

star.py --transform "[[-0.990457,-0.00727857,0.137627,352.06],[-0.00724027,0.999973,0.000778896,-45.7555],[-0.137629,-0.000224997,-0.990484,426.689]]" cryosparc_P3_J13_010_particles_mindist35A.star cryosparc_P3_J13_010_particles_mindist35A_moved1.star

subparticles.py --transform "[[-0.990457,-0.00727857,0.137627,352.06],[-0.00724027,0.999973,0.000778896,-45.7555],[-0.137629,-0.000224997,-0.990484,426.689]]" cryosparc_P3_J13_010_particles_mindist35A.star cryosparc_P3_J13_010_particles_mindist35A_moved1c.star --boxsize 400 --apix 0.95

I noticed that in either case, positions of particles in star file (rlnCoordinateX,Y and rlnOriginX/YAngst) did not change, so shifts were not applied. Moreover, reconstructed orientations did not match one another. With the second method, it seems that a 180 degree rotation around Y, plus a translation is required to bring the new volume in the correct position.

I double-checked my transformation matrix in Chimera using matrixset, which placed the volume precisely where it needs to be. So the problem is with the script or with the way I am trying to use it (?). BTW, are these two scripts expected to produce different results with the same transformation matrix? My guess is that star.py expects a matrix from ZYZ Euler sequence and subpartilcles.py expected a matrix from ZXZ (Chimera) - is this interpretation correct?

Our pyem installation is recent, from June this year (after star.py update). I did try an older version also, with the same result.

Peter

[pherepanov]

I think I found a workaround using star.py:

1. convert the Chimera-computed matrix from ZXZ (Chimera) to ZYZ (Relion) convention.
2. rotate particles with star.py:

star.py --transform "[[0.9999734855418153, 0.007240271878779304, -0.0007788943920751361],[0.007278570633670146, -0.9904573829780646, 0.1376270137501123],[0.0002249952962242808, -0.13762903388226003, -0.9904838203674767]]" cryosparc_P3_J13_010_particles_mindist35A.star cryosparc_P3_J13_010_particles_mindist35A_lower_subparticle.star

3. reconstruct rotated volume
4. measure required shift in Chimera (disable any rotations during fitting)
5. shift rotated particles with relion_star_handler via reciprocal shift of origins:

relion_star_handler --i cryosparc_P3_J13_010_particles_mindist35A_lower_subparticle.star --o cryosparc_P3_J13_010_particles_mindist35A_lower_subparticle_shifted_down.star --center --center_X 8.28719 --center_Y 53.3376 --center_Z -23.0831

Since this approach worked, I think there has to be a bug in subparticles.star, which does not get the right rotation around Y (either adds an extra 180 or does not do the main rotation).

Also, I noticed star.py adds extra columns to output star file (ucsfImageOriginalIndex, etc). Is there any use for them?

[asarnow]

It's probably just not updating rlnOriginAngstX/Y after modifying rlnOriginX/Y. I'll check this.

Rotation matrices and Euler angles are different representations, there's no "ZXZ" etc conventions for rotation matrices. So not sure what "conversion" you've done, could you explain what you mean? Have you done anything besides export the particles from cryosparc? (Everything except this step has been done in cryoSPARC?).

FYI the extra fields don't matter anymore, but they shouldn't have been written, this is fixed in the latest commit.

[pherepanov]

You are right matrix should not change. It has been a while, I do not remember what I did to the Chimera-computed matrix to make it work with star.py. The original star file was correct: reconstruction with the original star file generated the expected volume. Reconstruction with transformed star file generated a volume aligned as in Chimera. Local refinement of expanded particles worked as expected ... And no, nothing was done to star file other than conversion from cs to star.

But for the life of me I was not able to use the original Chimera matrix with either star.py or subparticles.py to do that. Reconstruction with transformed star file would have the volme nowhere near it has to be. Moreover, the two scripts (star.py and subparticles.py) produced different outputs. There is a comment, where you hint on a difference between the two scripts, possibly specifically related to chimera (https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=CCPEM;27b4f0db.1906) without details. Should there be a difference? Perhaps definition of the rotation matrix is different (coordinate system vs object rotation)?

[asarnow]

star.py and subparticles.py are different in that the former simply applies the matrix provided, while the latter assumes you have a matrix from ChimeraX fitmap (or measure rotation - I think matrixget was in camera coordinates not model coordinates) using MRC files with the origin as 0,0,0 / unset in the header (which is typical for cryo-EM maps, there is no popular software which outputs files with a non-zero origin in the header). That is also why the tool requires a boxsize (or a manually specified origin) - of course the particle metadata have no implied boxsize but the transformation you got from ChimeraX is relative to the 0,0,0 voxel of that particular map grid. I spent a lot of time drawing diagrams and testing to make sure it was right, so I could do multi-body refinements before there was a tool in Relion. (Though I can't say that I invented the idea first - Sjors already named it in a 2016 book chapter, before there was any implementation available, and I was also inspired by the Oxford local_rec program).

The map.py tool also has a --transform option that is matched to subparticles.py. You should be able to get a rotation from fitmap and apply that to the map, or to a star file and reconstruct, and end up with aligned volumes.

If you reconstruct the original exported cryoSPARC particles is the result in the same orientation as the cryoSPARC volume?

I just checked and it is the case that the transform function operates on the pixel origins, and then drops them when the Relion 3.1+ compatible star file is written out. I'll have subparticles.py sync the Angstrom coordinates to the pixel ones after the transformation.

[asarnow]

Syncing origins fixed in 01c4649

[pherepanov]

Thank you Daniel, we will update pyem and I will report how it works!
(in the meantime I was trying to retrace my steps I did to the original matrix.. The result of my misguided claculations was an approximate inversion of the rotation reported by Chimera, which reconstructed particles correctly when using star.py. I will try again with updated version).

[pherepanov]

Hi Daniel,
The new version still does not update origins in star file. But it also does not write the extra columns (_ucsf...) , which confirms the update has been applied. I will send you an example of my star file and the matrix I am trying to use.

[pherepanov]

Hi Daniel, Here is a link to my star file: https://drive.google.com/file/d/1ZL14YYp3R00gC1SMTdY9ggEjCUB2XCiM/view?usp=sharing Here is an example of a command with matrix from Chimera that superposes particle on itself with a correct rotation and shift to align subparticles: subparticles.py --transform [[-0.990502,-0.00737052,0.137304,352.149],[-0.00733846,0.999973,0.000739726,-45.7362],[-0.137306,-0.000274899,-0.990529,426.653]] cryosparc_P3_J13_010_particles_mindist35A.star cryosparc_P3_J13_010_particles_mindist35A_moved_1_testnew.star --boxsize 400 --apix 0.95 Here is matrix as printed by Chimera: Model 1.0 -0.990502 -0.00737052 0.137304 352.149 -0.00733846 0.999973 0.000739726 -45.7362 -0.137306 -0.000274899 -0.990529 426.653 I will be happy to share particle images, but at this stage it would be good to see origins move in the output text file. Peter

…

On Thu, Oct 17, 2024 at 7:48 AM Daniel Asarnow ***@***.***> wrote: It's probably just not updating rlnOriginAngstX/Y after modifying rlnOriginX/Y. I'll check this. Rotation matrices and Euler angles are different representations, there's no "ZXZ" etc conventions for rotation matrices. So not sure what "conversion" you've done, could you explain what you mean? Have you done anything besides export the particles from cryosparc? (Everything except this step has been done in cryoSPARC?). FYI the extra fields don't matter anymore, but they shouldn't have been written, this is fixed in the latest commit. — Reply to this email directly, view it on GitHub <#123 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHSHHOCMZYX6FQ3ADIVJVM3Z35MUXAVCNFSM6AAAAABN3JN77KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYGY3DMNRZHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>