What happened:
SingleTaskLearning._train() unconditionally assigns
os.environ["BASE_MODEL_URL"] = initial_model where initial_model is
obtained via kwargs.get("initial_model_url"). When initial_model_url
is absent from the algorithm YAML (a documented optional field in the
LLM-agent example), this value is None. Assigning None to os.environ
raises:
TypeError: str expected, not NoneType
Additionally, when multiple test cases run sequentially in the same
process via testcase.py, a stale BASE_MODEL_URL from a prior test
case that had initial_model_url set can persist into subsequent test
cases that omit it, causing the wrong pretrained checkpoint to be
loaded silently without any error.
What you expected to happen:
When initial_model_url is absent from the YAML, BASE_MODEL_URL should
not be set. If a prior test case had set it, the value should be
cleared before the next test case runs to prevent cross-test-case
environment pollution.
How to reproduce it:
-
Use an algorithm YAML with initial_model_url commented out:
basemodel:
url: ...
initial_model_url: (omitted)
-
Run any single-task learning benchmark:
python ianvs/main.py -f benchmarkingjob.yaml
-
Observe the crash:
TypeError: str expected, not NoneType
at singletask_learning.py line 120
Anything else we need to know:
Files involved:
Proposed fix — guard the assignment in _train():
if initial_model is not None:
os.environ["BASE_MODEL_URL"] = initial_model
else:
os.environ.pop("BASE_MODEL_URL", None)
The pop with a default clears any stale value from a prior test case.
What happened:
SingleTaskLearning._train() unconditionally assigns
os.environ["BASE_MODEL_URL"] = initial_model where initial_model is
obtained via kwargs.get("initial_model_url"). When initial_model_url
is absent from the algorithm YAML (a documented optional field in the
LLM-agent example), this value is None. Assigning None to os.environ
raises:
TypeError: str expected, not NoneType
Additionally, when multiple test cases run sequentially in the same
process via testcase.py, a stale BASE_MODEL_URL from a prior test
case that had initial_model_url set can persist into subsequent test
cases that omit it, causing the wrong pretrained checkpoint to be
loaded silently without any error.
What you expected to happen:
When initial_model_url is absent from the YAML, BASE_MODEL_URL should
not be set. If a prior test case had set it, the value should be
cleared before the next test case runs to prevent cross-test-case
environment pollution.
How to reproduce it:
Use an algorithm YAML with initial_model_url commented out:
basemodel:
url: ...
initial_model_url: (omitted)
Run any single-task learning benchmark:
python ianvs/main.py -f benchmarkingjob.yaml
Observe the crash:
TypeError: str expected, not NoneType
at singletask_learning.py line 120
Anything else we need to know:
Files involved:
core/testcasecontroller/algorithm/paradigm/singletask_learning/singletask_learning.py
core/testcasecontroller/testcase/testcase.py
env pollution a real cross-test-case issue
Proposed fix — guard the assignment in _train():
The pop with a default clears any stale value from a prior test case.