TiCLS: Tightly Coupled Language Text Spotter

News

• TiCLS has been accepted to WACV 2026! (arXiv)

Installation

Python 3.8 + PyTorch 1.9.0 + CUDA 11.1 + Detectron2 (v0.6)

git clone https://github.com/knowledge-computing/TiCLS.git
cd ticls

conda create -n ticls python=3.8 -y
conda activate ticls

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html

pip uninstall -y urllib3
pip install urllib3==1.26.6
pip install transformers==4.46.3

python setup.py build develop

Model Weights

TiCLS Weights

Dataset	Pretrain/Finetune	Download URL
ICDAR 2015	Pretrain	Download Weights
ICDAR 2015	Finetune	Download Weights
TotalText	Pretrain	Download Weights
TotalText	Finetune	Download Weights

PLM Weights

Component	Download URL
PLM (Encoder&Decoder)	Download PLM Weights
PLM Decoder for TiCLS	Download PLM Decoder Weights

Dataset

TiCLS Dataset

For downloading the dataset required to train TiCLS, please refer to the DeepSolo repository. We provide the corresponding annotation files (.json) below for training TiCLS. For evaluation, please unzip the file below and place it under "./datasets".

Annotations	Download URL
Training	Download Annotations
Evaluation	Download Annotations

PLM Dataset

Component	Download URL
PLM Tokenizer	Download Tokenizer
PLM Train Dataset	Download Train
PLM Test Dataset	Download Test

How to Run TiCLS

Train

python tools/train_net.py --config-file ${CONFIG_FILE} --num-gpus 4

Evaluate

python tools/train_net.py --config-file ${CONFIG_FILE} --num-gpus 2 --eval-only MODEL.WEIGHTS ${MODEL_PATH}

How to Run TiCLS from Scratch

We provide implementation details for training TiCLS from scratch.

Step 1. Pretrain LM (Encoder and decoder)

python plm_train/pretrain_LM_for_scenetext.py

Step 2. Detach PLM decoder for TiCLS initialization

python plm_train/get_decoder_from_PLM.py

Step 3. Pretrain TiCLS (Spotter)

Before training TiCLS, please make sure to correctly update the placeholder of the PLM weight (from Step 1) and its decoder weight (from Step 2) under ./adet/modeling/model/language.py

python tools/train_net.py --config-file ${CONFIG_FILE} --num-gpus 4

Step 4. Finetune TiCLS (Spotter)

python tools/train_net.py --config-file ${CONFIG_FILE} --num-gpus 2 --eval-only MODEL.WEIGHTS ${MODEL_PATH}