Image size should be even error

[jamesmkrieger]

Describe the bug
Running preprocessing without --no-keep-real makes particles with 65 pixels instead of 64, and then train_cv for opusDSD complains that the pixel size isn't even.

Traceback (most recent call last):
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 1157, in <module>
    main(args)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 725, in main
    data = dataset.LazyMRCData(args.particles, norm=args.norm,
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py", line 51, in __init__
    assert ny % 2 == 0, "Image size must be even"
AssertionError: Image size must be even

Looking at the particles in Python confirms the problem:

In [1]: particles='/home/jkrieger/ScipionUserData/projects/TestOpusDsd/Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt'

In [2]: from cryodrgn import dataset
Installed qt5 event loop hook.

In [3]: norm=None

In [4]: real_data=True

In [5]: invert_data=True

In [6]: ind=None

In [7]: use_real=True # it actually doesn't matter what value this takes as it isn't used in the code

In [8]: window=False

In [9]: relion31=True

In [10]: data=None

In [11]: datadir=None

In [12]: window_r=.85

In [13]: in_mem=True

In [14]: notinmem=False

In [15]:             data = dataset.LazyMRCData(particles, norm=norm,
    ...:                                        real_data=real_data, invert_data=invert_data,
    ...:                                        ind=ind, keepreal=use_real, window=False,
    ...:                                        datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[15], line 1
----> 1 data = dataset.LazyMRCData(particles, norm=norm,
      2                            real_data=real_data, invert_data=invert_data,
      3                            ind=ind, keepreal=use_real, window=False,
      4                            datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))

File ~/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py:51, in LazyMRCData.__init__(self, mrcfile, norm, real_data, keepreal, invert_data, ind, window, datadir, relion31, window_r, in_mem)
     49 ny, nx = particles[0].get().shape
     50 assert ny == nx, "Images must be square"
---> 51 assert ny % 2 == 0, "Image size must be even"
     52 log('Loaded {} {}x{} images'.format(N, ny, nx))
     53 self.particles = particles

AssertionError: Image size must be even

In [18]: mrcfile=particles

In [19]: particles = dataset.load_particles(mrcfile, True, datadir=datadir, relion31=relion31)

In [20]: type(particles)
Out[20]: list

In [21]:         N = len(particles)
    ...:         ny, nx = particles[0].get().shape

In [22]: ny
Out[22]: 65

In [23]: nx
Out[23]: 65

To Reproduce

CUDA_VISIBLE_DEVICES=0
python -m cryodrgn.commands.preprocess Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.mrcs  -D 64 --window-r 0.85 --max-threads 16  --relion31  -b 5000
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_ctf_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl  --relion31  -D 64  --Apix 3.54  --kv 300.0  --cs 2.7  -w 0.1  --ps 0
python -m cryodrgn.commands.train_cv Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt --poses Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl --ctf Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl --zdim 2 -o Runs/000237_OpusDsdProtTrain/output  -n 3 --preprocessed --max-threads 1  --enc-layers 3 --enc-dim 1024 --dec-layers 3 --dec-dim 1024 --lazy-single  --pe-type vanilla  --encode-mode grad  --template-type conv  -b 5000 --lr 0.00012  --beta-control 1.0  --beta cos  --downfrac 0.5  --valfrac 0.2  --lamb 1.0  --bfactor 4.0  --templateres 192

Expected behavior
I'm not actually sure. Somehow, I'd expect preprocess to work upstream of train_cv but maybe not with the arguments that I used as I just saw that it's not actually one of the recommended steps on the README. I suppose the convolutional network means that we don't have such a great need to downsample the map for efficiency anymore.

I think there is probably a problem that --no-keep-real doesn't do what it's supposed to (see #4, which I closed because I wasn't sure this is the right answer).

Another thing that I'd expect is that the keepreal argument does something in LazyMRCData and handles whether there is a check for pixel size needing to be even.

Additional context

• You should probably know that I am making a plugin for opusDSD within the Scipion workflow engine, which can be found at https://github.com/scipion-em/scipion-em-opusdsd. This allows opusDSD to be run from a GUI and included in pipelines. If you would like to meet and see what I'm doing and be involved, you are very welcome to.
• These results are from using the test dataset that was also used for CryoDRGN, which comes from a refinement in a Relion tutorial dataset and contains 1799 particles.

[jamesmkrieger]

I also have a warning from cryodrgn that we need to have a box size divisible by 8. Is this still true for opusdsd?

[jamesmkrieger]

I can actually note that I still get this error if I don't do any downsampling

[alncat]

I also have a warning from cryodrgn that we need to have a box size divisible by 8. Is this still true for opusdsd?

This is not true for opusDSD since I didn't implement apex acceleration. CryoDRGN implements an apex.amp acceleration which requires that kind of box size if you enable it during training.

[alncat]

Describe the bug Running preprocessing without --no-keep-real makes particles with 65 pixels instead of 64, and then train_cv for opusDSD complains that the pixel size isn't even.

Traceback (most recent call last):
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 1157, in <module>
    main(args)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 725, in main
    data = dataset.LazyMRCData(args.particles, norm=args.norm,
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py", line 51, in __init__
    assert ny % 2 == 0, "Image size must be even"
AssertionError: Image size must be even

Looking at the particles in Python confirms the problem:

In [1]: particles='/home/jkrieger/ScipionUserData/projects/TestOpusDsd/Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt'

In [2]: from cryodrgn import dataset
Installed qt5 event loop hook.

In [3]: norm=None

In [4]: real_data=True

In [5]: invert_data=True

In [6]: ind=None

In [7]: use_real=True # it actually doesn't matter what value this takes as it isn't used in the code

In [8]: window=False

In [9]: relion31=True

In [10]: data=None

In [11]: datadir=None

In [12]: window_r=.85

In [13]: in_mem=True

In [14]: notinmem=False

In [15]:             data = dataset.LazyMRCData(particles, norm=norm,
    ...:                                        real_data=real_data, invert_data=invert_data,
    ...:                                        ind=ind, keepreal=use_real, window=False,
    ...:                                        datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[15], line 1
----> 1 data = dataset.LazyMRCData(particles, norm=norm,
      2                            real_data=real_data, invert_data=invert_data,
      3                            ind=ind, keepreal=use_real, window=False,
      4                            datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))

File ~/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py:51, in LazyMRCData.__init__(self, mrcfile, norm, real_data, keepreal, invert_data, ind, window, datadir, relion31, window_r, in_mem)
     49 ny, nx = particles[0].get().shape
     50 assert ny == nx, "Images must be square"
---> 51 assert ny % 2 == 0, "Image size must be even"
     52 log('Loaded {} {}x{} images'.format(N, ny, nx))
     53 self.particles = particles

AssertionError: Image size must be even

In [18]: mrcfile=particles

In [19]: particles = dataset.load_particles(mrcfile, True, datadir=datadir, relion31=relion31)

In [20]: type(particles)
Out[20]: list

In [21]:         N = len(particles)
    ...:         ny, nx = particles[0].get().shape

In [22]: ny
Out[22]: 65

In [23]: nx
Out[23]: 65

To Reproduce

CUDA_VISIBLE_DEVICES=0
python -m cryodrgn.commands.preprocess Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.mrcs  -D 64 --window-r 0.85 --max-threads 16  --relion31  -b 5000
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_ctf_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl  --relion31  -D 64  --Apix 3.54  --kv 300.0  --cs 2.7  -w 0.1  --ps 0
python -m cryodrgn.commands.train_cv Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt --poses Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl --ctf Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl --zdim 2 -o Runs/000237_OpusDsdProtTrain/output  -n 3 --preprocessed --max-threads 1  --enc-layers 3 --enc-dim 1024 --dec-layers 3 --dec-dim 1024 --lazy-single  --pe-type vanilla  --encode-mode grad  --template-type conv  -b 5000 --lr 0.00012  --beta-control 1.0  --beta cos  --downfrac 0.5  --valfrac 0.2  --lamb 1.0  --bfactor 4.0  --templateres 192

Expected behavior I'm not actually sure. Somehow, I'd expect preprocess to work upstream of train_cv but maybe not with the arguments that I used as I just saw that it's not actually one of the recommended steps on the README. I suppose the convolutional network means that we don't have such a great need to downsample the map for efficiency anymore.

I think there is probably a problem that --no-keep-real doesn't do what it's supposed to (see #4, which I closed because I wasn't sure this is the right answer).

Another thing that I'd expect is that the keepreal argument does something in LazyMRCData and handles whether there is a check for pixel size needing to be even.

Additional context

• You should probably know that I am making a plugin for opusDSD within the Scipion workflow engine, which can be found at https://github.com/scipion-em/scipion-em-opusdsd. This allows opusDSD to be run from a GUI and included in pipelines. If you would like to meet and see what I'm doing and be involved, you are very welcome to.
• These results are from using the test dataset that was also used for CryoDRGN, which comes from a refinement in a Relion tutorial dataset and contains 1799 particles.

James, thank you very much! I will look into this issue!

[alncat]

Describe the bug Running preprocessing without --no-keep-real makes particles with 65 pixels instead of 64, and then train_cv for opusDSD complains that the pixel size isn't even.

Traceback (most recent call last):
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jkrieger/software/miniconda/envs/opusdsd-0.3.2b/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 1157, in <module>
    main(args)
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/commands/train_cv.py", line 725, in main
    data = dataset.LazyMRCData(args.particles, norm=args.norm,
  File "/home/jkrieger/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py", line 51, in __init__
    assert ny % 2 == 0, "Image size must be even"
AssertionError: Image size must be even

Looking at the particles in Python confirms the problem:

In [1]: particles='/home/jkrieger/ScipionUserData/projects/TestOpusDsd/Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt'

In [2]: from cryodrgn import dataset
Installed qt5 event loop hook.

In [3]: norm=None

In [4]: real_data=True

In [5]: invert_data=True

In [6]: ind=None

In [7]: use_real=True # it actually doesn't matter what value this takes as it isn't used in the code

In [8]: window=False

In [9]: relion31=True

In [10]: data=None

In [11]: datadir=None

In [12]: window_r=.85

In [13]: in_mem=True

In [14]: notinmem=False

In [15]:             data = dataset.LazyMRCData(particles, norm=norm,
    ...:                                        real_data=real_data, invert_data=invert_data,
    ...:                                        ind=ind, keepreal=use_real, window=False,
    ...:                                        datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[15], line 1
----> 1 data = dataset.LazyMRCData(particles, norm=norm,
      2                            real_data=real_data, invert_data=invert_data,
      3                            ind=ind, keepreal=use_real, window=False,
      4                            datadir=datadir, relion31=relion31, window_r=window_r, in_mem=(not notinmem))

File ~/software/scipion3/software/em/opusdsd-0.3.2b/cryodrgn/dataset.py:51, in LazyMRCData.__init__(self, mrcfile, norm, real_data, keepreal, invert_data, ind, window, datadir, relion31, window_r, in_mem)
     49 ny, nx = particles[0].get().shape
     50 assert ny == nx, "Images must be square"
---> 51 assert ny % 2 == 0, "Image size must be even"
     52 log('Loaded {} {}x{} images'.format(N, ny, nx))
     53 self.particles = particles

AssertionError: Image size must be even

In [18]: mrcfile=particles

In [19]: particles = dataset.load_particles(mrcfile, True, datadir=datadir, relion31=relion31)

In [20]: type(particles)
Out[20]: list

In [21]:         N = len(particles)
    ...:         ny, nx = particles[0].get().shape

In [22]: ny
Out[22]: 65

In [23]: nx
Out[23]: 65

To Reproduce

CUDA_VISIBLE_DEVICES=0
python -m cryodrgn.commands.preprocess Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.mrcs  -D 64 --window-r 0.85 --max-threads 16  --relion31  -b 5000
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_pose_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl  --relion31  -D 64  --Apix 3.54
python -m cryodrgn.commands.parse_ctf_star Runs/000186_OpusDsdProtPreprocess/extra/input_particles.star  -o Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl  --relion31  -D 64  --Apix 3.54  --kv 300.0  --cs 2.7  -w 0.1  --ps 0
python -m cryodrgn.commands.train_cv Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt --poses Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl --ctf Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl --zdim 2 -o Runs/000237_OpusDsdProtTrain/output  -n 3 --preprocessed --max-threads 1  --enc-layers 3 --enc-dim 1024 --dec-layers 3 --dec-dim 1024 --lazy-single  --pe-type vanilla  --encode-mode grad  --template-type conv  -b 5000 --lr 0.00012  --beta-control 1.0  --beta cos  --downfrac 0.5  --valfrac 0.2  --lamb 1.0  --bfactor 4.0  --templateres 192

Expected behavior I'm not actually sure. Somehow, I'd expect preprocess to work upstream of train_cv but maybe not with the arguments that I used as I just saw that it's not actually one of the recommended steps on the README. I suppose the convolutional network means that we don't have such a great need to downsample the map for efficiency anymore.

I think there is probably a problem that --no-keep-real doesn't do what it's supposed to (see #4, which I closed because I wasn't sure this is the right answer).

Another thing that I'd expect is that the keepreal argument does something in LazyMRCData and handles whether there is a check for pixel size needing to be even.

Additional context

• You should probably know that I am making a plugin for opusDSD within the Scipion workflow engine, which can be found at https://github.com/scipion-em/scipion-em-opusdsd. This allows opusDSD to be run from a GUI and included in pipelines. If you would like to meet and see what I'm doing and be involved, you are very welcome to.
• These results are from using the test dataset that was also used for CryoDRGN, which comes from a refinement in a Relion tutorial dataset and contains 1799 particles.

The preprocess routine in cryoDRGN might not work with opusDSD without special attention. OpusDSD takes the image stacks as input directly, and the image are in real space as it is. In contrast, cryoDRGN needs to take the fourier transform of the image, so they implement a preprocess routine to do the FFT in advance. Therefore, you need to make sure that the images downsampled by the preprocess of cryoDRGN is still in real space (I will check the no-keep-real argument). To downsample the image stack, I usually use the relion_preprocess from RELION, or specify a downsample argument during training, like --downsample 0.5, which will downsample the image to half of the original dimension. To go under the hood, James, you may note that there is a "data_augmentation" function in the train_cv.py, which handles the downsample, shift, and blurring, operations.

[alncat]

O, can you check the content of 'particles.64.ft.txt', is it pointing to particles.64.mrcs? You can also check the header of mrcs using IMOD's header command, like 'header particles.64.mrcs' .

[alncat]

https://github.com/alncat/opusDSD/blob/e931522987ed2b8fc8914943768d5a7452189493/cryodrgn/commands/preprocess.py#L107C21-L107C21 looks like no-keep-real will enable HT transform on the image, so the output is D+1

[jamesmkrieger]

Thanks for all the responses. That’s really helpful!

I’ll have a look at particles.64.ft.txt and see. I’d guess the images are already in Fourier space for that

[jamesmkrieger]

Yes, inside particles.64.ft.txt it says particles.64.0.ft.mrcs

[alncat]

Yes, inside particles.64.ft.txt it says particles.64.0.ft.mrcs

Sorry for the late reply, I am traveling last weekend. O, you can then try loading particles.64.mrcs directly to see if the images are in real space and in even size. Regarding the training command 'python -m cryodrgn.commands.train_cv Runs/000186_OpusDsdProtPreprocess/output_particles/particles.64.ft.txt --poses Runs/000186_OpusDsdProtPreprocess/output_particles/poses.pkl --ctf Runs/000186_OpusDsdProtPreprocess/output_particles/ctfs.pkl --zdim 2 -o Runs/000237_OpusDsdProtTrain/output -n 3 --preprocessed --max-threads 1 --enc-layers 3 --enc-dim 1024 --dec-layers 3 --dec-dim 1024 --lazy-single --pe-type vanilla --encode-mode grad --template-type conv -b 5000 --lr 0.00012 --beta-control 1.0 --beta cos --downfrac 0.5 --valfrac 0.2 --lamb 1.0 --bfactor 4.0 --templateres 192'. --preprocssed --max-threads 1 --enc-layers 3 --enc-dim 1024 --dec-layers 3 --dec-dim 1024' should be dropped as they are related to cryoDRGN. 'b' represents the batch size, 5000 might be too large. You can set it to a number around 20 that can fit into the gpu memory (depends on your hardware). If you have multiple gpu, you can then try 'multigpu' and 'num-gpus'. Since 64 is a very small size, downfrac can be set 1.0 (which means no downsampling), templateres can be set to smaller size like 128. Finally, you can try to make a mask using conesus model, following the link https://relion.readthedocs.io/en/release-3.1/SPA_tutorial/Mask.html . You can also try '--plot' option, which will show some intermediate results interactively during training.

[jamesmkrieger]

Thanks for the comments. No worry about the delay.

I think that particles.64.0.ft.mrcs is probably in Fourier space and not even size.

I decided to upsample the particles to 130 now and use downfrac 1.0 because there is an error that it has to be bigger than 128 and that's the first bigger even number. This is still for testing quickly at the moment and it is giving results now, which I think look ok considering how small the data set is.

I'll try removing those parts especially --preprocessed --max-threads 1 and change the batch size. Isn't good to have an option for controlling layers and dim for enc and dec?

Many thanks again

[alncat]

opusDSD/cryodrgn/commands/train_cv.py

Line 948 in 86bed17

assert args.downfrac*(D-1) >= 128

This line can be deleted since this size limit no longer holds. The intermediate tensors will be resampled to 12^3 in encoder,

opusDSD/cryodrgn/models.py

Line 663 in 86bed17

enc1 = F.interpolate(enc1, size=self.intermediate_size, mode="trilinear", align_corners=ALIGN_CORNERS) # 12^3

. Hence, the encoder works with any size now 😺. You can try to delete that line and test on 64x64 images. It will be great if we can control the number of layers, but this requires us to refactor the encoder and convtemplate classes ( this will make the code more readable btw).

[jamesmkrieger]

Ok, yes, I’ll delete this line and try it with 64x64 and remove the terms about the layers etc

[jamesmkrieger]

Thanks again for all your help