Sporadic divergence while playing replays with set seed

[Gerhard-Wonner]

Hello @ondras,

I implemented a replay-mode in my game. Therefore I store the seed of the game into the database together with the sequence of all the keystrokes of the player. The replay works quite well in most cases. :-)

My problem is: It works in most cases, but it does not work in all cases! :-(
Sometimes I realize that the replay diverges from the true game from a certain point (sometimes after thousands of turns). In most cases this results in the player-character eventually running into a wall. In rarer cases the whole level is generated in a different way.

I already separated the random value generation to different instances of the generator to keep the generation clean. E.g. I randomize the music from time to time, which I do with a separate instance of the randomizer. The remaining calls of the main-randomizer should now be time-independent and completely deterministic (with a given seed).

This is a very severe problem to me since I have no idea how to debug it. I tried a lot of things including logging the RNG-state and recording the game while playing and comparing it to the replay afterwards. My problem is, there is just too much information to keep track of. Logging everything is just not possible.

I am just a hobbyist and no professional JavaScript-developer, so I do not know many tricks or tools. Maybe there is a debugging tool out there, that could help me?

How would you approach this thing?

Thank you for your help!

Kind regards
Gerhard

[ondras]

Hi @Gerhard-Wonner,

thanks for the bugreport. You are not the only one running into these mysterious desyncs: it has been reported recently (see #201).

The issue is puzzling though, because the RNG itself is very simple and there are not many places where stuff could go wrong. I reviewed the code many times - and while the math aspect is out of my league (the underlying algorithm was provided by a real mathematician), managing the state and seeding is very straightforward. So even if we managed to reduce and find the core cause of the behavior, fixing it might be a completely different beast.

To further analyze the problem, I would suggest the following:

1. are you executing your JS server-side or client-side? If client-side, are you always using the same os/browser?
2. are you certain there are no un-controlled accesses to your RNG? The ROT.RNG object is used in a singleton-like manner in many places throughout the rot.js code, so it might be better to create a separate instance (via clone() and pass it around). You can then invalidate the global RNG (by some rogue assignment like ROT.RNG.getUniform = console.error) and see whether there si some code path using it accidentaly.
3. do you have a particular seed that always exhibits a desync? If so, we might simply try generating a very large number of random values and comparing the final value/state.

For instance, I just created a small testbed at https://jsfiddle.net/ondras/0Lgxbotw/. Here is what it shows on my Firefox:

Does your environment result in identical values?

[Gerhard-Wonner]

Hi @ondras,

thank you for your quick reply!

Regarding your point 1.: Good point to check the browser-compatibility first, since I execute my JS on the client-side. As a first test I picked out a replay of which I know that it runs the entire game on Chrome with no issues. I now also tried it with Firefox and Opera and it also worked there. The performance on Firefox was a little worse and it did not play sound-effects on Opera (but maybe I have to allow the page to play sound-effects first), but the game was finished on all browsers with a score of 86.

But maybe we should also try it on another computer since it is executed client-side. Could you please test the replay on your computer?
Just follow the following link and it will play:
https://runtothestairs.com/debugging-github-rotjs-203/index.php?replaygameid=170&alternativereplaymode

Does it also reach a score of 86 on your PC?

I will also address your other suggestions soon.

Kind regards
Gerhard

[ondras]

Does it also reach a score of 86 on your PC?

It does.

When I try doing stuff during the run (opening/closing devtools, switching tabs/windows, ...), the replay often freezes. The console then shows statistics not loaded - wait and try again! repeatedly, until killed/reloaded.

I was trying to verify whether open/closed devtools might influence the RNG, because fiddling with devtools often forbids many optimizations (JIT, AOT, ...) that the browser might be doing with JS. I have not been able to prove anything so far (if the run ends sucessfully without the mentioned error, the score is 86).

[ondras]

It might be useful to know (approximately) how many times the RNG is called in your case. You can measure that by instrumenting/patching the getUniform method (ideally for the global RNG as well as for any potential clones you might be using).

The number of calls, along with the value of a problematic seed, might be a good starting point for further investigations.

[Gerhard-Wonner]

Regarding your point 2.: I would love to do that, but I have no idea how to pass a cloned RNG to my ROT.Map.Digger. That is the reason, why I use the ROT.RNG instance for all sync-tasks and only have a cloned RNG for my async-tasks. Could you please tell me the syntax and I will change it.

I also run your testbed mentioned above and it yielded the same result as shown on your screenshot.

I will address your other suggestions soon.

[Gerhard-Wonner]

Good point with the message statistics not loaded - wait and try again!. It comes from a function that is not necessary when playing replays. I do now suppress calling that function when playing replays. So you already helped me finding a bug! :-)

[ondras]

Regarding your point 2.: I would love to do that, but I have no idea how to pass a cloned RNG to my ROT.Map.Digger.

Right, there is currently no way to do that - see comment #201 (comment) for reference.