The test results of each run are diffirent. Is this a bug?

[guojianwei]

I run test from saved model, but the SR/ABSR of each run is very different.
In src/dialogue_system/agent/agent.py, the _build_action_space method feasible_actions are Initialized from slot_set and disease_symptom which are loaded from pickle. So the action order in action_sapce may be different.
I modified the code in
line 262: for slot in sorted(self.slot_set.keys()):
line 269: for disease in sorted(self.disease_symptom.keys()):
it worked for me.

[LiuQL2]

Thank you very much for pointing this out and fixing it. I always thought it was caused by the randomness of the user simulator. Thanks again!

[ares5221]

I run test from saved model, but the SR/ABSR of each run is very different.
In src/dialogue_system/agent/agent.py, the _build_action_space method feasible_actions are Initialized from slot_set and disease_symptom which are loaded from pickle. So the action order in action_sapce may be different.
I modified the code in
line 262: for slot in sorted(self.slot_set.keys()):
line 269: for disease in sorted(self.disease_symptom.keys()):
it worked for me.

halo, i have some question when run test,
ValueError: The passed save_path is not a valid checkpoint: ./../model/dqn/checkpoint/checkpoint_d4_agt1_dqn1/model_d4_agent1_dqn1_s0.619_r18.221_t4.266_wd0.0_e432.ckpt
did you have same question

[AndreaMarino94]

Eseguo il test dal modello salvato, ma l'SR/ABSR di ogni corsa è molto diverso.
In src/dialogue_system/agent/agent.py, il metodo _build_action_space feasible_actions viene inizializzato da slot_set e disease_symptom che vengono caricati da pickle. Quindi l'ordine delle azioni in action_sapce potrebbe essere diverso.
Ho modificato il codice nella
riga 262: for slot in sorted(self.slot_set.keys()):
riga 269: for disease in sorted(self.disease_symptom.keys()):
ha funzionato per me.

alone, ho qualche domanda quando si esegue il test, ValueError: il save_path passato non è un checkpoint valido: ./../model/dqn/checkpoint/checkpoint_d4_agt1_dqn1/model_d4_agent1_dqn1_s0.619_r18.221_t4.266_wd0.0_e432.ckpt hai avuto la stessa domanda

you can take the saved_model path, from checkpoint.txt in "src\dialogue_system\model\dqn\checkpoint04"