"glia analyze (filename) integrity" fails with "json.decoder.JSONDecodeError"

[atthewmay]

Issue Description

Running "glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity" gives a "json.decoder.JSONDecodeError" in the try-except block of create_epl_gen_v2 in glia/eyecandy.py.

In my eye-candy directory, I ran "docker-compose up" prior to running the above command.

I have performed "git pull" for the eye-candy directory and the glia directory. Tried while on the "master" branch of glia and the 3brain_units branch.

Steps to reproduce.

1. docker-compose up to start eyecandy, using the attached docker-config.yml.
2. Run "glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity", using the attached config.yaml file (for manual thresholding).

What's the Actual Result?

Copied below are 1. the error message from glia, and 2. The error message from eyecandy.

1. From Glia
   (glia) matthewhunt@Matthews-MacBook-Pro 200818 % glia analyze --configuration config.yaml R1_E3_AMES_540min_ffhq_BnW_6100_9200 integrity
   version 0.5.1
   Analyzing R1_E3_AMES_540min_ffhq_BnW_6100_9200
   No .stim file found. Creating from .analog file.
   couldn't parse json from <Response [500]>
   Traceback (most recent call last):
   File "/Users/matthewhunt/miniconda3/envs/glia/bin/glia", line 11, in
   load_entry_point('glia', 'console_scripts', 'glia')()
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 829, in call
   return self.main(*args, **kwargs)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 782, in main
   rv = self.invoke(ctx)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
   return _process_result(sub_ctx.command.invoke(sub_ctx))
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1268, in invoke
   Command.invoke(self, ctx)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
   return ctx.invoke(self.callback, **ctx.params)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/core.py", line 610, in invoke
   return callback(*args, **kwargs)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
   return f(get_current_context(), *args, **kwargs)
   File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia_scripts/command_line.py", line 343, in analyze
   metadata, stimulus_list = glia.create_stimuli(
   File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia/eyecandy.py", line 220, in create_stimuli
   metadata, stimulus_gen = create_epl_gen_v2(program, epl, window_width,
   File "/Users/matthewhunt/Coding/Seattle_Research/MEA_Project/glia/glia/eyecandy.py", line 71, in create_epl_gen_v2
   response = r.json()
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/site-packages/requests/models.py", line 898, in json
   return complexjson.loads(self.text, **kwargs)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/init.py", line 357, in loads
   return _default_decoder.decode(s)
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/decoder.py", line 337, in decode
   obj, end = self.raw_decode(s, idx=_w(s, 0).end())
   File "/Users/matthewhunt/miniconda3/envs/glia/lib/python3.8/json/decoder.py", line 355, in raw_decode
   raise JSONDecodeError("Expecting value", s, err.value) from None
   json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

2. The error message from eyecandy:
   nginx_1 | 172.17.0.1 - - [02/Sep/2020:22:24:17 +0000] "POST /analysis/run-program HTTP/1.1" 500 21 "-" "python-requests/2.24.0" "-"
   web_1 | <-- POST /analysis/run-program
   web_1 | analysis/run-program
   web_1 | compiling EPL.
   web_1 | xxx POST /analysis/run-program 500 540ms -
   web_1 |
   web_1 | ReferenceError: measureIntegrity is not defined
   web_1 | at vm.js:39:1
   web_1 | at Script.runInContext (vm.js:107:20)
   web_1 | at VM.run (/www/node_modules/vm2/lib/main.js:204:72)
   web_1 | at compileJSProgram (/www/src/epl/eval.js:43:8)
   web_1 | at router.post.ctx (/www/src/app.js:268:13)
   web_1 | at dispatch (/www/node_modules/koa-router/node_modules/koa-compose/index.js:44:32)
   web_1 | at next (/www/node_modules/koa-router/node_modules/koa-compose/index.js:45:18)
   web_1 | at /www/node_modules/koa-router/lib/router.js:346:16
   web_1 | at dispatch (/www/node_modules/koa-router/node_modules/koa-compose/index.js:44:32)
   web_1 | at /www/node_modules/koa-router/node_modules/koa-compose/index.js:36:12
   web_1 |

[atthewmay]

The config.yml for the "glia analyze" looks like:

The docker-compose.yml looks like this

[tbenst]

Thanks for creating this issue! I’ll take a closer look tomorrow but in the meantime a couple issues:

• the calibration file should have 6 values (2 for each stimulus_index % 3—the three “Eye of Sauron” modes in histogram)
• In future please post the relevant lab notebook. You can upload a gist if you would like. Lab notebook for this experiment can be found by searching for the name “ R1_E3_AMES_540min_ffhq_BnW_6100_9200“ and contains important fields for troubleshooting, particularly the epl
• your error is because the EPL is failing to compile. In particular, EyeCandy used to define measureIntegrity in every EPL file. This has now been added as a function in the global namespace of the language, per https://github.com/tbenst/eye-candy/pull/31/files. See generators.js. This suggests that your eyecandy is not the latest version. If you have not restarted your docker container after pulling and rebuilding, please do so

If that does not solve the issue, please let me know

[atthewmay]

Also, in case this is confusing, I had to rename the experiment to be "R1_E3_AMES_540min_ffhq_BnW_6100_9200" rather "than R1_E3_AMES_540min_ffhq_BnW_6100_9201", bc it seemed the lab_notebook.yml had 6100_9200.

…

________________________________ From: Tyler Benster <notifications@github.com> Sent: Wednesday, September 2, 2020 8:40 PM To: tbenst/glia <glia@noreply.github.com> Cc: Hunt, Matthew <huntm@wustl.edu>; Author <author@noreply.github.com> Subject: Re: [tbenst/glia] "glia analyze (filename) integrity" fails with "json.decoder.JSONDecodeError" (#29) Thanks for creating this issue! I’ll take a closer look tomorrow but in the meantime a couple issues: * the calibration file should have 6 values (2 for each stimulus_index % 3—the three “Eye of Sauron” modes in histogram) * In future please post the relevant lab notebook. You can upload a gist if you would like. Lab notebook for this experiment can be found by searching for the name “ R1_E3_AMES_540min_ffhq_BnW_6100_9200“ and contains important fields for troubleshooting, particularly the epl * your error is because the EPL is failing to compile. In particular, EyeCandy used to define measureIntegrity in every EPL file. This has now been added as a function in the global namespace of the language, per https://github.com/tbenst/eye-candy/pull/31/files. See generators.js. This suggests that your eyecandy is not the latest version. If you have not restarted your docker container after pulling and rebuilding, please do so If that does not solve the issue, please let me know — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#29 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHTEQ2RJNLIO3HONDMFWGZTSD3X2RANCNFSM4QUAIFSA>.

________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.