PySpark - starting from Spark 2.4.1 python jobs don't work

[dos65]

It seems that from 2.4.1 using authentification token for py4j GatewayServer became a requirement.
Now python jobs are failing with the following error:

You are trying to pass an insecure Py4j gateway to Spark. This is not allowed as it is a security risk