ASQuery: A Query-Based Model For Action Segmentation

Introduction

This repository provides a PyTorch implementation of the paper "ASQuery: A Query-Based Model For Action Segmentation".

Installation

Follow INSTALL.md for installing necessary dependencies.

To Reproduce Our Results on Breakfast

Prepare dataset

• Download dataset refer to MS-TCN.
• Unpack the file under ./data (or elsewhere and link to ./data).
• The folder structure should look like

This folder
│   README.md
│   ...  
│
└───data/
│    └───breakfast/
│    │	 └───features/
│    │	 └───groundTruth/
│    │	 └───splits/
│    │	 └───mapping.txt  
│    └───...
|
└───libs
│
│   ...

Training and Evaluation

• To train our ASQuery. This will create an experiment folder under ./ckpt that stores training config, logs, and checkpoints.

python main.py configs/breakfast/config.yaml --output result

• [Optional] Monitor the training using TensorBoard

tensorboard --logdir=./ckpt/breakfast/config_result/logs

• Evaluate the trained model.

python eval.py configs/breakfast/config.yaml ckpt/breakfast/config_result

Acknowledgement

ActionFormer: The main codebase we built upon and it is an wonderful model.