Skip to content

Joint model V1, V2, train + test scripts, and smoke tests#26

Open
Pranav1006 wants to merge 29 commits into
release_2.0from
joint-model
Open

Joint model V1, V2, train + test scripts, and smoke tests#26
Pranav1006 wants to merge 29 commits into
release_2.0from
joint-model

Conversation

@Pranav1006

Copy link
Copy Markdown
Collaborator

Summary

This PR includes two versions of the joint model. The first version uses the joint architecture and optimizes a weighted combination of the reconstruction and VC losses. The second version optimizes the same loss while preserving the optimization of each VC term for explainability (the main model to use for testing). There is also a train + test script that includes smoke tests showing that training steps and gradients work.

Major Changes

  • Joint Model V1
  • Joint Model V2
  • Joint Model V2 train script
  • Joint Model V2 data processing
  • Smoke tests

Future Considerations

We should double-check whether the optional disc losses are reached by the gradients, and try different training times/amounts of layers, since the VC loss might take longer to converge. Hyperparameters are easily configurable, so we can run different combinations/ablations.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rewrite this file so that it modular across datasets. I will upload AMIGOS to the drive in order to do this.

…epare datasets to generated AMIGOS files from csv, and added SLURM scripts for running training scripts
@Pranav1006

Copy link
Copy Markdown
Collaborator Author

joint_v2_data.py was rewritten to load datasets from a config class, and instances were created for DREAMER and AMIGOS. Additionally, prepare_datasets.py from stsnet was modified and used to convert amigos into the .npy files needed to load all data. Two minimal slurm scripts were added for running the training script, one for arousal and one for valence.

During data preparation, the script detected NaN values in the CSV file, so the data/labels for AMIGOS may contain NaN values. If no nan values are found in the AMIGOS CSV, then we can go back and inspect the data loading script to see if it somehow manipulates values and turns them into NaNs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants