Skip to content
This repository was archived by the owner on Jul 30, 2019. It is now read-only.
This repository was archived by the owner on Jul 30, 2019. It is now read-only.

python killed - out of memory during registration #24

Description

@DGalt

We've been testing ndreg as an alternative registration package for our Clarity imaging efforts, but we're running in to an issue with the registration failing due to an OOM error in what seems to be the last stage of the registration.

The tutorial / example data provided runs fine - this is failing on our data. The first several steps seems to complete as expected:

Step 0: alpha=0.05, beta=0.05, scale=0.0625
        E, E_velocity, E_rate, E_image (E_image %), LearningRate
0.      26814.1, 62.2496, 0, 26751.8 (87.9049%), 5.500000e-03
1.      26397.5, 78.2021, 0, 26319.3 (86.4834%), 1.512500e-03
2.      25807.2, 95.4081, 0, 25711.7 (84.4872%), 1.663750e-03
3.      24154.6, 121.688, 0, 24032.9 (78.9707%), 1.830125e-03
4.      22683.3, 155.63, 0, 22527.7 (74.0246%), 2.013138e-03
5.      20909.5, 202.948, 0, 20706.6 (68.0405%), 2.214451e-03
6.      20573.7, 233.436, 0, 20340.3 (66.837%), 1.217948e-03
7.      20334.7, 268.579, 0, 20066.1 (65.9359%), 1.339743e-03
8.      19753.9, 289.534, 0, 19464.4 (63.9586%), 7.368587e-04
9.      19294.7, 312.791, 0, 18981.9 (62.3734%), 8.105445e-04
10.     19167.6, 326.595, 0, 18841 (61.9104%), 4.457995e-04
11.     19000.9, 341.867, 0, 18659.1 (61.3125%), 4.903794e-04
12.     18915.4, 359.172, 0, 18556.3 (60.9747%), 5.394174e-04
13.     18887.1, 378.791, 0, 18508.3 (60.8171%), 5.933591e-04
14.     18867.5, 400.87, 0, 18466.6 (60.6802%), 6.526950e-04
15.     18848, 425.853, 0, 18422.1 (60.534%), 7.179645e-04
16.     18847.8, 454.67, 0, 18393.1 (60.4386%), 7.897610e-04
E = 18847.8 (60.4386%)
Length = 20.1142
Time = 8.62127s (0.143688m)

Step 1: alpha=0.05, beta=0.05, scale=0.125
        E, E_velocity, E_rate, E_image (E_image %), LearningRate
0.      28493.2, 65.7145, 0, 28427.5 (97.5929%), 5.500000e-03
1.      27603.1, 59.5668, 0, 27543.6 (94.5585%), 7.562500e-04
2.      27411.8, 56.9109, 0, 27354.9 (93.9108%), 8.318750e-04
3.      27277, 57.2933, 0, 27219.7 (93.4468%), 9.150625e-04
4.      27067.2, 61.968, 0, 27005.2 (92.7103%), 1.006569e-03
5.      26750.5, 72.5735, 0, 26678 (91.5868%), 1.107226e-03
6.      26592.5, 87.0846, 0, 26505.4 (90.9945%), 1.217948e-03
7.      26565.4, 102.462, 0, 26463 (90.8487%), 1.339743e-03
8.      26564.9, 110.63, 0, 26454.3 (90.819%), 7.368587e-04
9.      26562.8, 120.35, 0, 26442.4 (90.7782%), 8.105445e-04
10.     26420, 131.428, 0, 26288.6 (90.25%), 8.915990e-04
11.     26406.2, 134.154, 0, 26272.1 (90.1934%), 2.451897e-04
12.     26402.6, 134.51, 0, 26268.1 (90.1797%), 3.371359e-05
13.     26400.8, 134.707, 0, 26266.1 (90.1729%), 1.854247e-05
14.     26400.8, 134.71, 0, 26266.1 (90.1729%), 3.186987e-07
15.     26400.8, 134.71, 0, 26266.1 (90.1729%), 2.191054e-08
16.     26400.8, 134.71, 0, 26266.1 (90.1729%), 1.205080e-08
17.     26400.8, 134.71, 0, 26266.1 (90.1729%), 6.627938e-09
E = 26411.1 (90.2084%)
Length = 19.269
Time = 9.69856s (0.161643m)

Step 2: alpha=0.05, beta=0.05, scale=0.25
        E, E_velocity, E_rate, E_image (E_image %), LearningRate
0.      26447.6, 15.0638, 0, 26432.6 (94.9078%), 5.500000e-03
1.      26355.9, 14.3033, 0, 26341.6 (94.581%), 3.781250e-04
2.      26279.6, 14.1924, 0, 26265.5 (94.3077%), 4.159375e-04
3.      26159.6, 14.7062, 0, 26144.9 (93.8748%), 4.575313e-04
4.      26040.3, 15.8101, 0, 26024.5 (93.4425%), 5.032844e-04
5.      25829.6, 17.3934, 0, 25812.2 (92.6803%), 5.536128e-04
6.      25683.2, 19.2385, 0, 25663.9 (92.1479%), 6.089741e-04
7.      25631.8, 21.1908, 0, 25610.6 (91.9566%), 6.698715e-04
8.      25623.7, 23.3273, 0, 25600.4 (91.9197%), 7.368587e-04
9.      25623.7, 23.3942, 0, 25600.3 (91.9194%), 2.532952e-05
10.     25623.7, 23.4126, 0, 25600.3 (91.9193%), 6.965617e-06
11.     25623.7, 23.4128, 0, 25600.3 (91.9193%), 5.986077e-08
12.     25623.7, 23.4128, 0, 25600.3 (91.9193%), 1.646171e-08
E = 25623.7 (91.9193%)
Length = 8.34867
Time = 10.7544s (0.17924m)

Step 3: alpha=0.05, beta=0.05, scale=0.5
        E, E_velocity, E_rate, E_image (E_image %), LearningRate
0.      35048.8, 7.49631, 0, 35041.3 (99.3777%), 5.500000e-03
1.      34209.6, 4.8356, 0, 34204.7 (97.005%), 3.025000e-03
2.      34100, 7.29589, 0, 34092.7 (96.6872%), 1.663750e-03
3.      34081.6, 8.84533, 0, 34072.8 (96.6308%), 1.830125e-03
4.      34053.3, 11.4954, 0, 34041.8 (96.543%), 2.013138e-03
5.      33971, 14.1881, 0, 33956.8 (96.3019%), 2.214451e-03
6.      33951, 18.4157, 0, 33932.6 (96.2331%), 2.435896e-03
7.      33867.6, 22.7148, 0, 33844.9 (95.9844%), 2.679486e-03
8.      33836.3, 26.0582, 0, 33810.3 (95.8864%), 1.473717e-03
9.      33792.9, 29.1741, 0, 33763.7 (95.7543%), 1.621089e-03
10.     33760.4, 33.28, 0, 33727.1 (95.6506%), 1.783198e-03
11.     33718.6, 37.7322, 0, 33680.9 (95.5193%), 1.961518e-03
12.     33696.3, 43.4092, 0, 33652.9 (95.44%), 2.157670e-03
13.     33651.8, 49.7127, 0, 33602.1 (95.296%), 2.373436e-03
14.     33636.2, 57.863, 0, 33578.3 (95.2285%), 2.610780e-03
15.     33522.6, 66.4359, 0, 33456.2 (94.8822%), 2.871858e-03
16.     33522.6, 66.4386, 0, 33456.2 (94.8822%), 7.712510e-07
E = 33522.6 (94.8822%)
Length = 11.1824
Time = 41.7592s (0.695987m)

Step 4: alpha=0.05, beta=0.05, scale=1.0
        E, E_velocity, E_rate, E_image (E_image %), LearningRate
0.      31299.2, 1.72881, 0, 31297.4 (97.985%), 2.750000e-03
1.      31128.1, 0.965935, 0, 31127.2 (97.4519%), 1.512500e-03
2.      30994.7, 1.89466, 0, 30992.8 (97.0314%), 1.663750e-03
3.      30984.2, 2.91225, 0, 30981.3 (96.9954%), 1.830125e-03
4.      30983.3, 3.05556, 0, 30980.2 (96.9919%), 2.516422e-04
5.      30983.3, 3.05556, 0, 30980.2 (96.9919%), 1.689492e-08
E = 30983.3 (96.9919%)
Length = 2.2311
Time = 155.781s (2.59635m)

However, after Step 4 the memory usage blows up (during steps 0-4 monitoring with docker stats memory usage stays below 4GB, but then balloons to use all available memory, >130GB) and the process is killed.

[  687.083739] Out of memory: Kill process 3752 (python) score 957 or sacrifice child
[  687.090101] Killed process 3752 (python) total-vm:1323215624kB, anon-rss:137628016kB, file-rss:0kB, shmem-rss:0kB
[  701.215907] oom_reaper: reaped process 3752 (python), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Full output from dmesg is available here

Docker Info:

Version 2.0.0.3 (31259)
Channel: stable
Build: 8858db3

This is being run on Windows 10.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions