Mismatch between the number of Agents and Neighbors

[mattiacolucci]

Hi, I am trying to use trajdata with Trajectron++ and I am encountering some problems regarding the batches. I am using nuScenes dataset and my goal is to predict for all the agents in the scene. For this reason, I am using the dataset in the scene-ccentric version and the convert it to AgentBatch using the provided function by your library. The problem is that for some samples, the number of agents does not match the number of neighbors. Specifically, given a sample, the shape[0] of neigh_hist (representing the agents in the scene) is different from its shape[1] (representing the neighbors). This means that not all agents are included in the batch from the scene. Supporting this, by visualizing the Agent Batch using your own visualization method, the image generated shows more agents than those available in agent_hist and neigh_hist.

Can you tell me how to access all the agent's data in the scene from the batch?

I am using the same dataset configuration as yours in the batch example provided.

Here is the way with which I load the dataset:

def convert_dataset_agent_batch(dataset: UnifiedDataset):
    agent_batches = []
    for idx, sample in tqdm(enumerate(dataset), desc="Converting dataset to AgentBatch format...", total=len(dataset), ncols=80):
        agent_batch = convert_to_agent_batch(
            sample,
            only_types=[AgentType.VEHICLE],
            no_types=[AgentType.UNKNOWN],
            agent_interaction_distances=attention_radius,
            incl_map=hyperparams["map_encoding"],
            map_params=map_params,
            max_neighbor_num=None,
            state_format=None,
            standardize_data=True,
            standardize_derivatives=False,
            pad_format="right",
        )
        agent_batches.append(agent_batch)

    return agent_batches

And here is how I convert to Agent Batch:

agent_batch = convert_to_agent_batch(
            sample,
            only_types=[AgentType.VEHICLE],
            no_types=[AgentType.UNKNOWN],
            agent_interaction_distances=attention_radius,
            incl_map=hyperparams["map_encoding"],
            map_params=map_params,
            max_neighbor_num=None,
            state_format=None,
            standardize_data=True,
            standardize_derivatives=False,
            pad_format="right",
        )

I precise that in the sample considered, the vehicles in addition that are showed in the image, are vehicles (not pedestrians or other kinds of agents).

The specific sample I am talking about the sample with index 916 of nuscene trainval-val dataset.

Thanks in advance.