Training doesn't run + how to predict with custom query?

[fromfuture-ai]

From the repository, I have run this on the command line. And the output was as following.

sqlova$ python bert/convert_tf_checkpoint_to_pytorch.py --td_checkpoint_path $BERT_BASE_DIR/bert_model.ckpt --bert_config_file $BERT_BASE_DIR/bert_config.json --pytorch_dump_path $BERT_BASE_DIR/pytorch_model.bin

Converting TensorFlow checkpoint from data_and_model/uncased_L-12_H-768_A-12/bert_model.ckpt
Loading bert/embeddings/LayerNorm/beta with shape [768]
Numpy array shape (768,)
Loading bert/embeddings/LayerNorm/gamma with shape [768]
Numpy array shape (768,)
Loading bert/embeddings/position_embeddings with shape [512, 768]
Numpy array shape (512, 768)
Loading bert/embeddings/token_type_embeddings with shape [2, 768]
Numpy array shape (2, 768)
Loading bert/embeddings/word_embeddings with shape [30522, 768]
Numpy array shape (30522, 768)

.
(Many more lines)
.
Loading encoder/layer_9/output/LayerNorm/gamma
Loading encoder/layer_9/output/dense/bias
Loading encoder/layer_9/output/dense/kernel
Loading pooler/dense/bias
Loading pooler/dense/kernel
Loading redictions/output_bias
Skipping
Loading redictions/transform/LayerNorm/beta
Skipping
Loading redictions/transform/LayerNorm/gamma
Skipping
Loading redictions/transform/dense/bias
Skipping
Loading redictions/transform/dense/kernel
Skipping
Loading eq_relationship/output_bias
Skipping
Loading eq_relationship/output_weights
Skipping

I have downloaded the file, and put it under data_and_model, and run this,
python3 train.py --seed 1 --bS 16 --accumulate_gradients 2 --bert_type_abb uS --fine_tune --lr 0.001 --lr_bert 0.00001 --max_seq_leng 222

Output was as following,
BERT-type: uncased_L-12_H-768_A-12
Batch_size = 32
BERT parameters:
learning rate: 1e-05
Fine-tune BERT: True
vocab size: 30522
hidden_size: 768
num_hidden_layer: 12
num_attention_heads: 12
hidden_act: gelu
intermediate_size: 3072
hidden_dropout_prob: 0.1
attention_probs_dropout_prob: 0.1
max_position_embeddings: 512
type_vocab_size: 2
initializer_range: 0.02
Load pre-trained parameters.
Seq-to-SQL: the number of final BERT layers to be used: 2
Seq-to-SQL: the size of hidden dimension = 100
Seq-to-SQL: LSTM encoding layer size = 2
Seq-to-SQL: dropout rate = 0.3
Seq-to-SQL: learning rate = 0.001

But, it didn’t generate any files like these,

• model_best.pt: the checkpoint of the the sequence-to-SQL module.
• model_bert_best.pt: the checkpoint of the BERT module.
  ◦ results_dev.jsonl: json file for official evaluation.

As you have mentioned. It also took only a few seconds to run.

1. What’s the problem here?

2. How can I actually run this, and finally test with my custom query?

[longxudou]

For question2, add --do_train args will work.