EDTER

EDTER: Edge Detection with Transformer
Mengyang Pu, Yaping Huang, Yuming Liu, Qingji Guan and Haibin Ling
CVPR 2022

Please refer to supplementary material for more results.

Usage

Our project is developed based on MMsegmentation. Please follow the official MMsegmentation INSTALL.md and getting_started.md for installation and dataset preparation.

Linux

The full script for setting up EDTER with conda is following here.

conda create -n edter python=3.7 -y
conda activate edter
conda install pytorch=1.6.0 torchvision cudatoolkit=10.1 -c pytorch -y
pip install mmcv-full==1.2.2 -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
cd EDTER
pip install -e .  # or "python setup.py develop"
pip install -r requirements/optional.txt

Datasets

BSDS500

Download the augmented BSDS500 data (1.2GB) from here.

|-- data
    |-- BSDS
        |-- ImageSets
        |   |-- train_pair.txt
        |   |-- test.txt
        |   |-- pascal_train_pair.txt
        |-- train
        |   |-- aug_data
        |   |-- aug_data_scale_0.5
        |   |-- aug_data_scale_1.5
        |   |-- aug_gt
        |   |-- aug_gt_scale_0.5
        |   |-- aug_gt_scale_1.5
        |-- test
        |   |-- 2018.jpg
        ......

PASCAL VOC

Download the augmented PASCAL VOC data from here (Code:83cv).

|-- data
    |-- PASCAL
        |-- ImageSets
        |   |-- pascal_train_pair.txt
        |   |-- test.txt
        |-- aug_data
            |-- 0.0_0
            |   |-- 2008_000002.jpg
            ......
            |-- 0.0_1
            |   |-- 2008_000002.jpg
            ......
        |-- aug_gt
            |-- 0.0_0
            |   |-- 2008_000002.png
            ......
            |-- 0.0_1
            |   |-- 2008_000002. png
            ......

NYUD

Download the augmented NYUD data (~11GB) from here(Code:t2ce).

|-- data
    |-- NYUD
        |-- ImageSets
        |   |-- hha-test.txt
        |   |-- hha-train.txt
        |   |-- image-test.txt
        |   |-- image-train.txt
        |-- train
            |-- GT
            |-- GT_05
            |-- GT_15
            |-- HHA
            |-- HHA_05
            |-- HHA_15
            |-- Images
            |-- Images_05
            |-- Images_15
        |-- test
            |-- HHA
            |   |-- img_5001.png
            ......
            |-- Images
            |   |-- img_5001.png
            ......

initial weights

If you are unable to download due to network reasons, you can download the initial weights from here(VIT-base-p16) and here(VIT-large-p16). The two .pth files of initial weights should be placed in the folder.

|-- EDTER
    |-- pretrain
        |-- jx_vit_base_p16_384-83fb41ba.pth
        |-- jx_vit_large_p16_384-b3be5167.pth

Training

Note: Our project only supports distributed training on multiple GPUs on one machine or a single GPU on one machine.

The training of Stage I on BSDS500

If you want to set the batch size in each GPU, please refer to https://github.com/MengyangPu/EDTER/blob/bbee219d5713a77aeec61c0f7fde93620cb02d60/configs/bsds/EDTER_BIMLA_320x320_80k_bsds_bs_8.py#L99 For example, data = dict(samples_per_gpu=4) means that each GPU can process 4 images. Therefore, the batch size of training = samples_per_gpu * GPU_NUM. In the experiments, we set the training batch size to 8, where samples_per_gpu=4 and GPU_NUM=2.

The command to train the first-stage model is as follows

cd EDTER
bash ./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} 
# For example, train Stage I on the BSDS500 dataset with 2 GPUs
cd EDTER
./tools/dist_train.sh configs/bsds/EDTER_BIMLA_320x320_80k_bsds_bs_8.py 2

The training of Stage II on BSDS500

Change the '--global-model-path' in train_local.py. https://github.com/MengyangPu/EDTER/blob/846370ece24b9dc8925037853ccfa33d6cadeaa2/tools/train_local.py#L22C27-L23C54

cd EDTER
./tools/dist_train_local.sh ${GLOBALCONFIG_FILE} ${CONFIG_FILE} ${GPU_NUM} 
# For example, train Stage II on the BSDS500 dataset with 2 GPUs
cd EDTER
./tools/dist_train_local.sh configs/bsds/EDTER_BIMLA_320x320_80k_bsds_bs_8.py configs/bsds/EDTER_BIMLA_320x320_80k_bsds_local8x8_bs_8.py 2

Testing

Single-scale testing

Change the '--config', '--checkpoint', and '--tmpdir' in test.py.

python tools/test.py

Multi-scale testing

Change the '--globalconfig', '--config', '--global-checkpoint', '--checkpoint', and '--tmpdir' in test_local.py.
Use the config file ending in _ms.py in configs/EDTER.

python tools/test_local.py

Eval

BSDS500

cd eval
run eval_bsds.m

NYUD

Download the matfile(NYUD) from here(Code:25p8).

cd eval
run eval_nyud.m

Results

If you want to compare your method with EDTER, you can download the precomputed results BSDS500 and NYUD(code:b941).

Download Pre-trained model.

model	Pre-trained Model
EDTER-BSDS-VOC-StageI	BaiDuNetdisk (Code:l282) or Google Drive
EDTER-BSDS-VOC-StageII	BaiDuNetdisk (Code:skjw) or Google Drive
EDTER-NYUD-RGB-StageI	BaiDuNetdisk (Code:dwdi)
EDTER-NYUD-RGB-StageII	BaiDuNetdisk (Code:s00u)
EDTER-NYUD-HHA-StageI	BaiDuNetdisk (Code:ko2f)
EDTER-NYUD-HHA-StageII	BaiDuNetdisk (Code:p7wu)

Important notes

• All the models are trained and tested on a single machine with multiple NVIDIA-V100-32G GPUs.
• Training on distributed GPUs is not supported.

Acknowledgments

• We thank the anonymous reviewers for valuable and inspiring comments and suggestions.
• Thanks to previous open-sourced repo:
  SETR
  MMsegmentation
  

Reference

@InProceedings{Pu_2022_CVPR,
    author    = {Pu, Mengyang and Huang, Yaping and Liu, Yuming and Guan, Qingji and Ling, Haibin},
    title     = {EDTER: Edge Detection With Transformer},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {1402-1412}
}