ImportError: No module named docker

[TG9541]

I tried running docker-readthedocs, but it looks like a docker related dependency is missing. I'm new to RTD, and hence I can't easily say how far off the Dockerfile in moul/docker-readthedocs is from building successfully.

Commands to reproduce the problem:

git clone https://github.com/moul/docker-readthedocs.git 
docker-compose run --service-ports --rm readthedocs

Here is the "tail" of the build log with the error message:

....
Step 21 : CMD /bin/rtd-start.sh
 ---> Running in e79da91191db
 ---> 4df232adf658
Removing intermediate container e79da91191db
Successfully built 4df232adf658
+ curl -XPUT http://elasticsearch:9200/readthedocs/
{"acknowledged":true}+ cd /app/readthedocs
+ ln -s ../manage.py .
+ PYTHON=/venv/bin/python
+ /venv/bin/python manage.py syncdb --noinput
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ /venv/bin/python manage.py migrate
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ echo 'from django.contrib.auth.models import User; User.objects.create_superuser('\''admin'\'', '\''admin@localhost'\'', '\''admin'\'')'
+ /venv/bin/python manage.py shell
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ /venv/bin/python manage.py loaddata test_data
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ /venv/bin/python manage.py makemessages --all
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ /venv/bin/python manage.py compilemessages
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
+ /venv/bin/python manage.py runserver 0.0.0.0:8000
+ /venv/bin/python manage.py celeryd -l INFO
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/venv/local/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/readthedocs/config.py", line 1, in <module>
    from readthedocs.settings.docker import *
ImportError: No module named docker

[moul]

Hi @TG9541, thanks for reporting,

I will give a look, I fear they updated their install method again.

I suggest you to pull the image I built 21 days ago: https://hub.docker.com/r/moul/readthedocs/ instead of rebuilding a new one

[TG9541]

Hi @moul, thanks, I'll try that. This sure is a moving target :-)

[TG9541]

@moul: I pulled the image above, changed the docker-compose.yml to use the image, and I still get the same error when I run the docker-compose line above.
Maybe I am supposed to add a "docker.py" for configuration?

[moul]

Ok perfect,

No you are not supposed to add this file by yourself, this file used to exist in the readthedocs codebase, here is where I hardcode import it: https://github.com/moul/docker-readthedocs/blob/master/config/etc/default/rtd-config.py

I just removed the deprecated import (automated docker build in progress)

Can you give a new try ?

[TG9541]

Thanks, that fixed it :-)

By the way, the docker-as-the-build-environment feature was what initially caught my interest. I'd have to pass-in the docker daemon socket (e.g. like -v /var/run/docker.sock:/var/run/docker.sock). Isn't the docker.py import the file where I should provide the configuration?

[moul]

@TG9541 happy I could help :)

I didn't heard about this docker-as-the-build-environment feature before,
but in general, to practice "docker in docker (a.k.a dind)", passing the -v /var/run/docker.sock:/var/run/docker.sock is the way to go.
If you only need to introspect the docker images, you can just pass -v /var/lib/docker:/var/lib/docker:ro in read-only, but in your case, I think you will require a full socket access

Good luck !