eval_conformal_figure.py fails due to missing wandb keys and dataframe columns

[purentap]

Hi,

I'm trying to run eval_conformal_figure.py to obtain the classification results, but I believe there may be an issue with the script. I attempted to debug it, but I’m still unable to resolve it, partly due to confusion around the naming conventions used in the code.

I’ve identified a few potential issues:

1. I believe the issue arises here:
   https://github.com/vla-safe/SAFE/blob/main/failure_prob/utils/wandb.py#L326-L329
   None of the keys in data_run_keys match k.replace(...) == logged_name, so the DataFrame is never saved.

2. Another issue appears here:
   https://github.com/vla-safe/SAFE/blob/main/scripts/get_wandb_metrics.py#L340
   The column method_name_full does not exist. I tried replacing it with method, but I’m not sure if this is the intended behavior.

3. Finally, here:
   https://github.com/vla-safe/SAFE/blob/main/scripts/get_wandb_metrics.py#L358
   The code fails because none of the expected columns are present in the DataFrame, likely because it is not being saved due to issue (1).

Any clarification or guidance would be appreciated.