Files in some directories end up owned by root, causing test failures

[MichaelClerx]

On fedora, using docker from the terminal.

Following first instruction here: https://github.com/Chaste/chaste-docker?tab=readme-ov-file#recommended-using-a-pre-built-image

docker run --init -it --rm -v chaste_data:/home/chaste chaste/release

It sets things up alright, but some of the files in the volume it creates are owned by root, causing test failures

chaste@172860a06b68:~$ ls -all
total 28
drwxr-x--- 1 chaste chaste  182 Jan  8 17:09 .
drwxr-xr-x 1 root   root     12 Dec 15 00:36 ..
-rw-r--r-- 1 chaste chaste  220 Jan  6  2022 .bash_logout
-rw-r--r-- 1 chaste chaste 3771 Jan  6  2022 .bashrc
drwxr-xr-x 1 chaste chaste    6 Jan  8 17:09 .cache
-rw-r--r-- 1 chaste chaste   67 Dec 15 04:49 .gitconfig
drwxr-xr-x 1 chaste chaste   12 Jan  8 17:09 .local
-rw-r--r-- 1 chaste chaste  807 Jan  6  2022 .profile
drwxr-xr-x 1 chaste chaste  554 Jan  8 17:09 build
lrwxrwxrwx 1 chaste chaste   18 Dec 15 00:37 lib -> /home/chaste/build
drwxr-xr-x 1 chaste chaste   76 Jan  8 17:09 output
lrwxrwxrwx 1 chaste chaste   25 Dec 15 00:37 projects -> /home/chaste/src/projects
drwxr-xr-x 1 chaste chaste  170 Jan  8 17:09 scripts
drwxr-xr-x 1 chaste chaste  774 Jan  8 17:09 src
lrwxrwxrwx 1 chaste chaste   19 Dec 15 00:37 testoutput -> /home/chaste/output
chaste@172860a06b68:~$ ls src -all
total 248
drwxr-xr-x 1 chaste chaste    774 Jan  8 17:09 .
drwxr-x--- 1 chaste chaste    182 Jan  8 17:09 ..
-rw-r--r-- 1 root   root     2648 Nov 23  2023 .clang-format
-rw-r--r-- 1 root   root     4472 Nov 23  2023 .clang-tidy
-rw-r--r-- 1 root   root     1983 Nov 23  2023 .codecov.yml
drwxr-xr-x 1 root   root       52 Jan  8 17:09 .devcontainer
drwxr-xr-x 1 root   root      190 Jan  8 17:09 .git
drwxr-xr-x 1 root   root       46 Jan  8 17:09 .github
-rw-r--r-- 1 root   root     1089 Feb 21  2024 .gitignore
-rw-r--r-- 1 root   root        0 Nov 23  2023 .gitmodules
-rw-r--r-- 1 root   root     2355 Oct 16 16:09 CITATION.cff
-rw-r--r-- 1 root   root     3660 Sep  7 09:54 CITATION.md

[MichaelClerx]

Steps to reproduce:

1. Ensure there's no volume called chaste_data already docker volume rm chaste_data
2. docker run --init -it --rm -v chaste_data:/home/chaste chaste/release, creating a new chaste_data in the process
3. cd build; ctest -R TestCardiacSimulation$ -V

445: Entering TestMono1dSodiumBlockBySettingNamedParameter
445: -- The C compiler identification is GNU 11.4.0
445: -- The CXX compiler identification is GNU 11.4.0
445: -- Detecting C compiler ABI info
445: -- Detecting C compiler ABI info - done
445: -- Check for working C compiler: /usr/bin/cc - skipped
445: -- Detecting C compile features
445: -- Detecting C compile features - done
445: -- Detecting CXX compiler ABI info
445: -- Detecting CXX compiler ABI info - done
445: -- Check for working CXX compiler: /usr/bin/c++ - skipped
445: -- Detecting CXX compile features
445: -- Detecting CXX compile features - done
445: -- Found Python3: /usr/bin/python3.10 (found suitable version "3.10.12", minimum required is "3.5") found components: Interpreter 
445: -- Found Chaste libraries in /home/chaste/build
445: -- Found Python3: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
445: -- Configuring done
445: -- Generating done
445: -- Build files have been written to: /home/chaste/build/heart/dynamic/tmp_1478_1736430574
445: [ 33%] Processing CellML file ../../../../src/heart/dynamic/luo_rudy_1991_dyn.cellml
445: Traceback (most recent call last):
445:   File "/home/chaste/build//codegen_python3_venv/bin/chaste_codegen", line 8, in <module>
445:     sys.exit(chaste_codegen())
445:   File "/home/chaste/build/codegen_python3_venv/lib/python3.10/site-packages/chaste_codegen/_command_line_script.py", line 249, in chaste_codegen
445:     process_command_line()
445:   File "/home/chaste/build/codegen_python3_venv/lib/python3.10/site-packages/chaste_codegen/_command_line_script.py", line 223, in process_command_line
445:     write_file(file, code)
445:   File "/home/chaste/build/codegen_python3_venv/lib/python3.10/site-packages/chaste_codegen/_script_utils.py", line 20, in write_file
445:     file = open(file_name, 'w')
445: PermissionError: [Errno 13] Permission denied: '/home/chaste/src/heart/dynamic/luo_rudy_1991_dyn.hpp'
445: gmake[2]: *** [CMakeFiles/luo_rudy_1991_dyn.dir/build.make:74: /home/chaste/src/heart/dynamic/luo_rudy_1991_dyn.hpp] Error 1
445: gmake[1]: *** [CMakeFiles/Makefile2:82: CMakeFiles/luo_rudy_1991_dyn.dir/all] Error 2
445: gmake: *** [Makefile:91: all] Error 2
445: 
445: /home/chaste/src/heart/test/TestCardiacSimulation.hpp:193: Error: Test failed: 
445: Chaste error: ./heart/src/odes/CellMLToSharedLibraryConverter.cpp:259: Conversion of CellML to Chaste shared object failed. Error was: 
445: Chaste error: ./heart/src/odes/CellMLToSharedLibraryConverter.cpp:226: Error executing command: system(cmake --build . --config Debug)
445: Failed

where the pertinent bit is

PermissionError: [Errno 13] Permission denied: '/home/chaste/src/heart/dynamic/luo_rudy_1991_dyn.hpp'

[MichaelClerx]

chaste-docker/Dockerfile

Line 107 in 691f0ef

USER ${USER}

...

chaste-docker/Dockerfile

Line 140 in 691f0ef

RUN build_chaste.sh ${GIT_TAG}

...

chaste-docker/scripts/build_chaste.sh

Line 27 in 691f0ef

git clone --recursive -b $VERSION $GIT_REMOTE $CHASTE_SOURCE_DIR

so don't see why it wouldn't end up owned by the chaste user... something to do with the volume?

[MichaelClerx]

In terms of UID it looks exactly the same:

chaste@2490cb6461dc:~$ ls -ln
total 12
drwxr-xr-x 1 1000 1000  574 Jan  9 13:47 build
lrwxrwxrwx 1 1000 1000   18 Dec 15 00:37 lib -> /home/chaste/build
drwxr-xr-x 1 1000 1000 9364 Jan  9 14:00 output
lrwxrwxrwx 1 1000 1000   25 Dec 15 00:37 projects -> /home/chaste/src/projects
drwxr-xr-x 1 1000 1000  170 Jan  9 13:47 scripts
drwxr-xr-x 1 1000 1000  774 Jan  9 13:47 src
lrwxrwxrwx 1 1000 1000   19 Dec 15 00:37 testoutput -> /home/chaste/output
chaste@2490cb6461dc:~$ ls -ln ./src/
total 228
-rw-r--r-- 1 0 0   2355 Oct 16 16:09 CITATION.cff
-rw-r--r-- 1 0 0   3660 Sep  7 09:54 CITATION.md
-rw-r--r-- 1 0 0  46793 Nov 20 15:14 CMakeLists.txt
-rw-r--r-- 1 0 0   4610 Feb 22  2024 ChasteParameters.xml
-rw-r--r-- 1 0 0 127235 Dec 18  2023 Doxyfile
-rw-r--r-- 1 0 0   1734 Jan 10  2024 LICENSE
-rw-r--r-- 1 0 0   2946 Nov 20 15:14 README.md

where 0 maps to root both locally and in the docker
and where 1000 maps to me locally and chaste in the docker

[MichaelClerx]

Not too surprisingly, starting from chaste/base and running build_chaste.sh develop checks out the repo as chaste and files end up with the correct ownership and the tests pass

[MichaelClerx]

Using

docker run --init -it --rm -v chaste_data:/home/chaste chaste/develop

has the same permissions issue as release

[MichaelClerx]

Have now tried it on an ubuntu (running docker inside of virtualbox...) and it does exactly the same: the files in src are checked out as root.

Working hypothesis: The Dockerfile specifies that it should be root, and whatever is being done afterwards to make it work in docker desktop fixes this

[MichaelClerx]

I think I've found it:

chaste-docker/Dockerfile

Lines 143 to 162 in 340edc5

	# ------------------------------------------------------------------------------
	FROM base AS build
	# Build Chaste: GIT_TAG can be a branch or release ('-' skips by default)
	ARG GIT_TAG=-
	ENV GIT_TAG=${GIT_TAG}
	RUN build_chaste.sh ${GIT_TAG}
	# Automatically mount the home directory in a volume to persist changes made there.
	# NOTE: After declaring the volume, changes to the contents during build will not persist.
	VOLUME "${CHASTE_DIR}"
	# ------------------------------------------------------------------------------
	FROM build AS test
	# Optionally run a test suite before finalising the image.
	# NOTE: These test outputs will not appear in the volume.
	ARG TEST_SUITE=-
	ENV TEST_SUITE=${TEST_SUITE}
	RUN test.sh ${TEST_SUITE}

These lines create a whole new image, and the USER is not inherited from the parent image. So build is run as root, hence the checking out happens as root.

Solution is just to add another

USER ${USER}

inside the build and test images

[MichaelClerx]

Nope that's not it!

When I build a new one locally it works fine

Will wait until the new images become available and then try again

[bdevans]

Hi @MichaelClerx, thanks for the detailed investigations. The new develop and release images are now available (build on the new base image) so it might be worth trying them. I might also explicitly set the user in the subsequent build stages (e.g. after FROM build AS test) since the rules about user inheritance could have changed (or might yet) so that way it should be clear what's happening.

[MichaelClerx]

Interestingly, the release build has the same issue!

chaste@5c8b2508a66d:~/src$ ls -all
total 252
drwxr-xr-x 1 chaste chaste    790 Jan 15 19:46 .
drwxr-x--- 1 chaste chaste    144 Jan 15 19:46 ..
-rw-r--r-- 1 root   root     2648 Nov 23  2023 .clang-format
-rw-r--r-- 1 root   root     4472 Nov 23  2023 .clang-tidy
-rw-r--r-- 1 root   root     1983 Nov 23  2023 .codecov.yml
drwxr-xr-x 1 root   root       52 Jan 15 19:46 .devcontainer
drwxr-xr-x 1 root   root      190 Jan 15 19:46 .git
drwxr-xr-x 1 root   root       46 Jan 15 19:46 .github
-rw-r--r-- 1 root   root     1088 Dec 20 18:10 .gitignore
-rw-r--r-- 1 root   root        0 Nov 23  2023 .gitmodules
-rw-r--r-- 1 root   root     2355 Oct 16 16:09 CITATION.cff
-rw-r--r-- 1 root   root     3660 Sep  7 09:54 CITATION.md
-rw-r--r-- 1 root   root    49323 Jan  6 16:55 CMakeLists.txt
-rw-r--r-- 1 root   root     4610 Feb 22  2024 ChasteParameters.xml
-rw-r--r-- 1 root   root   127235 Dec 18  2023 Doxyfile
-rw-r--r-- 1 root   root     1734 Jan  6 16:55 LICENSE
-rw-r--r-- 1 root   root     2946 Nov 20 15:14 README.md
-rw-r--r-- 1 root   root     1869 Jan  6 16:55 SConstruct
drwxr-xr-x 1 root   root      244 Jan 15 19:46 anim
drwxr-xr-x 1 root   root       50 Jan 15 19:46 apps
-rw-r--r-- 1 root   root     1111 Nov 23  2023 archive_convert.sed
drwxr-xr-x 1 root   root       52 Jan 15 19:46 cell_based
-rw-r--r-- 1 root   root     7342 Nov 23  2023 chaste-legacy.supp
-rw-r--r-- 1 root   root     2988 Nov 23  2023 chaste.supp
-rw-r--r-- 1 root   root       32 Nov 23  2023 chaste_codegen.txt
drwxr-xr-x 1 root   root      508 Jan 15 19:46 cmake
drwxr-xr-x 1 root   root       52 Jan 15 19:46 continuum_mechanics
drwxr-xr-x 1 root   root       52 Jan 15 19:46 crypt
drwxr-xr-x 1 root   root      160 Jan 15 19:46 cxxtest
drwxr-xr-x 1 root   root       20 Jan 15 19:46 docker
drwxr-xr-x 1 root   root      206 Jan 15 19:46 docs
drwxr-xr-x 1 root   root       52 Jan 15 19:46 global
drwxr-xr-x 1 root   root       66 Jan 15 19:46 heart
drwxr-xr-x 1 root   root       52 Jan 15 19:46 io
drwxr-xr-x 1 root   root       20 Jan 15 19:46 lib
drwxr-xr-x 1 root   root       52 Jan 15 19:46 linalg
drwxr-xr-x 1 root   root       20 Jan 15 19:46 linklib
drwxr-xr-x 1 root   root       52 Jan 15 19:46 lung
drwxr-xr-x 1 root   root       52 Jan 15 19:46 mesh
drwxr-xr-x 1 root   root       52 Jan 15 19:46 ode
drwxr-xr-x 1 root   root       20 Jan 15 19:46 output
drwxr-xr-x 1 root   root       52 Jan 15 19:46 pde
drwxr-xr-x 1 root   root       32 Jan 15 19:46 projects
drwxr-xr-x 1 root   root       76 Jan 15 19:46 pychaste
drwxr-xr-x 1 root   root       92 Jan 15 19:46 python
-rw-r--r-- 1 root   root      628 Nov 23  2023 simple_cube_binary.ele
chaste@5c8b2508a66d:~/src$ cat LICENSE 
BSD 3-Clause License.

Copyright (c) 2005-2025, University of Oxford.
All rights reserved.

University of Oxford means the Chancellor, Masters and Scholars of the
University of Oxford, having an administrative office at Wellington
Square, Oxford OX1 2JD, UK.

This file is part of Chaste.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
 * Redistributions of source code must retain the above copyright notice,
   this list of conditions and the following disclaimer.
 * Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.
 * Neither the name of the University of Oxford nor the names of its
   contributors may be used to endorse or promote products derived from this
   software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

License says 2025, so looks like the new image to me

Other folders look OK, except for the projects folder because it's a symlink to a folder inside src