One common complaint about discrete-time active inference / pymdp is the fact that in a typical infer_and_plan() loop the agent plans at every timestep. This is often unnecessary, since the originally formulated plan may be valid for multiple timesteps and no re-planning is required. One feature would be to allow the agent to only intermittently plan (e.g., once every K timesteps, or once a given state-dependent criterion has been reached, like the free energy surpasses a threshold or an intermittent goal in a chain of waypoints has been accomplished).
Here are two ways this could be implemented within the current infer_and_plan() logic
- use a
lax.cond() call inside infer_and_plan() to conditionally plan (i.e., execute the policy_search Callable) based on some externally provided function like plan_criterion: Callable
- ask users to implement this conditional planning logic inside their custom
policy_search function which is passed into infer_and_plan(), and then it's on the burden of the user to determine what inputs policy_search takes and how it determines whether to actually execute the policy optimization method in question. This would require us to make policy_search's assumed signature more flexible so it can take more arguments (like last_plan or whatever the user needs to evaluate their continue-planning condition).
A third way is just not to implement this feature and place the onus on users to write their own custom rollout function if they want this sort of custom intermittent-planning functionality. I only mention it as a generic feature because I've heard multiple users ask whether pymdp can accommodate this.
One common complaint about discrete-time active inference / pymdp is the fact that in a typical
infer_and_plan()loop the agent plans at every timestep. This is often unnecessary, since the originally formulated plan may be valid for multiple timesteps and no re-planning is required. One feature would be to allow the agent to only intermittently plan (e.g., once every K timesteps, or once a given state-dependent criterion has been reached, like the free energy surpasses a threshold or an intermittent goal in a chain of waypoints has been accomplished).Here are two ways this could be implemented within the current
infer_and_plan()logiclax.cond()call insideinfer_and_plan()to conditionally plan (i.e., execute thepolicy_searchCallable) based on some externally provided function likeplan_criterion: Callablepolicy_searchfunction which is passed intoinfer_and_plan(), and then it's on the burden of the user to determine what inputspolicy_searchtakes and how it determines whether to actually execute the policy optimization method in question. This would require us to makepolicy_search's assumed signature more flexible so it can take more arguments (likelast_planor whatever the user needs to evaluate their continue-planning condition).A third way is just not to implement this feature and place the onus on users to write their own custom
rolloutfunction if they want this sort of custom intermittent-planning functionality. I only mention it as a generic feature because I've heard multiple users ask whether pymdp can accommodate this.