fix(agent): back Policies with a hashable tuple#416
Open
gauraaansh wants to merge 1 commit into
Open
Conversation
Agent.policies is marked static=True to support vmap/scan over batched Agents (infer-actively#227), but Policies currently stores policy_arr as a jnp.ndarray. As described in infer-actively#346, constructing equivalent Agents repeatedly causes eqx.filter_jit to miss its compilation cache. Store policies internally as a hashable nested tuple instead, exposing policy_arr via a property that materializes a JAX array on access. This preserves the static Agent.policies partition while allowing equivalent policy tables to hash by value. Constructing an Agent inside an active jax.jit trace required an additional change, since traced arrays cannot be converted back to concrete Python values. To avoid this, add pure-Python tuple builders for policy construction, and only convert to a JAX array at the property/return boundary. Fixes infer-actively#346
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #346
Agent.policiesis markedstatic=True(#345) to supportvmap/scanover a batched Agent (#227), sincepolicieshas no batch dimension. ButPolicies.policy_arris a realjnp.ndarray, so the static partition ends up identity-based instead of value-based, and every freshAgentconstruction is a cache miss undereqx.filter_jit.This backs
Policieswith a hashable tuple instead, exposingpolicy_arras a property that materializes a JAX array on access.Agent.policiesstays static, but the static partition is now hashable by value.Constructing an
Agentinside an activejax.jittrace broke the first version of this, since a traced array can't be converted to a concrete tuple mid-trace. Traced this toconstruct_policies()converting to JAX arrays earlier than necessary. Added pure-Python tuple builders that defer the JAX conversion to the property/return boundary.Testing
vmap/scan/slicing over a batched Agent still work (Agentpoliciesfield in equinox module has no batch_dim #227)