Declaring content/ path as a VOLUME mount point causing problems

[maxcct]

A while ago I opened this issue: #185, which was answered and closed before I had a chance to respond.

My problem is that this image prevents any modification of directories within content/, and I can't see why this is desirable or necessary. Probably I'm just being dense, but I can't work out how the proposed solution would enable me to use this image in a way that would make it possible to modify directories within content/.

Could the guidance be clarified/expanded, or, ideally, could the line declaring content/ as a fixed volume be removed? What useful purpose is it serving?

[pascalandy]

It's best practice to define the VOLUME in which the app is living in. Ghost lives inside /content per say.

[mason]

@pascalandy then how does one modify the content of the volume(say to add s3 storage adapter)? At the moment doing this doesn't work

FROM ghost:3.12.0

WORKDIR /var/lib/ghost

RUN npm install ghost-storage-adapter-s3 \
        && mkdir -p content/storage \
        && cp -vr ./node_modules/ghost-storage-adapter-s3 ./content/storage

declaring it as volume blows out that directory. https://docs.docker.com/engine/reference/builder/#volume

Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded.

[pascalandy]

I don't know. But here is where you can know :)
https://ghost.org/docs/concepts/storage-adapters/

[mason]

I don't know. But here is where you can know :)
https://ghost.org/docs/concepts/storage-adapters/

Am I missing something? I definitely checked that page before posting. It doesn't talk much about this issue. In fact, the solutions for s3 storage adapter does not work specifically for this reason.

[pascalandy]

As I said, I don't use storage adapters.

My gut feeling.

Install the adapter like your did. (I would add this step in the original dockerfile
You need to update your config_production.json file with the appropriate ENV.

Have you checked on https://forum.ghost.org/ ?

[yosifkit]

    && cp -vr ./node_modules/ghost-storage-adapter-s3 ./content/storage

But why do you need to move the adapter inside of ghost content? Shouldn't npm install (or yarn add) be enough for node and ghost to be able to use it? (just like the sqlite3 that gets installed in the dockerfile) Maybe it needs a gosu node npm install ghost-storage-adapter-s3 so that it is running as the correct user?

ghost/3/debian/Dockerfile

Line 74 in 20f003a

if ! gosu node yarn add "sqlite3@$sqlite3Version" --force; then \

---

If it really needs to be in the content/storage directory, then you can just drop it in $GHOST_INSTALL/content.orig/storage and it will be placed into the volume on first start:

ghost/3/debian/docker-entrypoint.sh

Lines 10 to 20 in 20f003a

	if [[ "$*" == node*current/index.js* ]]; then
	baseDir="$GHOST_INSTALL/content.orig"
	for src in "$baseDir"/*/ "$baseDir"/themes/*; do
	src="${src%/}"
	target="$GHOST_CONTENT/${src#$baseDir/}"
	mkdir -p "$(dirname "$target")"
	if [ ! -e "$target" ]; then
	tar -cC "$(dirname "$src")" "$(basename "$src")" | tar -xC "$(dirname "$target")"
	fi
	done
	fi

Related/duplicate of #212.

[yosifkit]

Now as to the content VOLUME which is the original question.

What useful purpose is it serving?

The way I understand it, the content/ of ghost is considered the mutable state of the application. Since docker does not copy the files in the image to a bind mount, we have to copy in the default files in the entrypoint. We have the volume to ensure all users don't store stuff there so that the resulting image can be used with any kind of volume for persistent storage.

This is basically the same as the WordPress image; although it has to do all of WordPress instead of just a subdirectory docker-library/wordpress#232 (comment).

[mason]

For anyone wondering, the following dockerfile worked for me

FROM ghost:3.12.0

WORKDIR /var/lib/ghost

RUN npm install ghost-storage-adapter-s3 \
        && mkdir -p ./content.orig/adapters/storage \
        && cp -vr ./node_modules/ghost-storage-adapter-s3 ./content.orig/adapters/storage/ghost-s3

Docker run:

docker run -p 2368:2368 -e storage__active=ghost-s3 -e storage__ghost-s3__accessKeyId=<keyid> -e storage__ghost-s3__secretAccessKey=<secret> -e storage__ghost-s3__bucket=<bucket> -e storage__ghost-s3__region=<region> ghost/s3:3.12.0

But why do you need to move the adapter inside of ghost content? Shouldn't npm install (or yarn add) be enough for node and ghost to be able to use it?

I think how it works is that the code explicitly imports from a specific directory. see here:
https://github.com/TryGhost/Ghost/blob/9d4b0c09a8061768b145a737fd0a8ec25108134d/core/server/api/v2/images.js#L1

[derek-adair]

This is a confusing issue to me; is this an issue with your development workflow or deploying in production? Is this in a custom docker image?

Seems like some additional advice on how to pre-seed content/plugins/config is needed in the README's and that it should be done like advised here by adding to content.orig.

This was actually a point of frustration for me when first leveraging this image -- it's not a real pleasant experience learning how this image works and will trip up people who are weak system admins.

[muratcorlu]

For anyone wondering, the following dockerfile worked for me

FROM ghost:3.12.0

WORKDIR /var/lib/ghost

RUN npm install ghost-storage-adapter-s3 \
        && mkdir -p ./content.orig/adapters/storage \
        && cp -vr ./node_modules/ghost-storage-adapter-s3 ./content.orig/adapters/storage/ghost-s3

Docker run:

docker run -p 2368:2368 -e storage__active=ghost-s3 -e storage__ghost-s3__accessKeyId=<keyid> -e storage__ghost-s3__secretAccessKey=<secret> -e storage__ghost-s3__bucket=<bucket> -e storage__ghost-s3__region=<region> ghost/s3:3.12.0

But why do you need to move the adapter inside of ghost content? Shouldn't npm install (or yarn add) be enough for node and ghost to be able to use it?

I think how it works is that the code explicitly imports from a specific directory. see here: TryGhost/Ghost@9d4b0c0/core/server/api/v2/images.js#L1

Does this still work with v5? Is that content.orig folder is a special one that Ghost checks over regular content folder? Or do you set as your content path? (If so, what is the point here?)

I'm also having the same issue as not being able to add/change storage adapters to the projects that allready has mounted content path as volume. Added or changed storage become inside the content/adapters/storage folder of docker image, but that path is overriden by the container's volume. I feel like we need to pick specific folders inside content path those are only changed by users, like themes or images, files etc. But not the adapters folder.

[muratcorlu]

I tried the approach putting adapter inside content.orig folder, but that doesn't help. In Dockerfile content.orig folder is overwritten with the content from volume. Check here:

ghost/5/alpine/Dockerfile

Line 48 in 4a77eba

mv "$GHOST_CONTENT" "$GHOST_INSTALL/content.orig"; \

So I can not update adapter. Only option that I see now is adding multiple volumes per each needed folder in content folder, like:

volumes:
  - settings:/var/lib/ghost/content/settings
  - themes:/var/lib/ghost/content/themes
  - data:/var/lib/ghost/content/data

[HighPriest]

I've just wasted an hour, trying to figure out, where I was overriding the content directory, because ghost could not find contents of my volume mounted to /var/lib/ghost...

Can we please stop with implicit definition of VOLUME $GHOST_CONTENT?
What is the point of it anyway? Why isn't /var/lib/ghost given a persistent volume, when the configuration files seem just as important as the media?

Edit:
I might be over-reacting. Maybe I don't need persistence of the configuration files, when they can be stored through ENV VARS.
I'll see if using a volume with just /content works for me and I'll come back to report.

Edit2:
I come to confirm. The implicit definition of VOLUME $GHOST_CONTENT is BAD. I would need manage a separate volume for the configuration file, because there is no other way to set things like:

"security": {
  "staffDeviceVerification": false
},

or

"logging": {
  "transports": [
    "stdout"
  ]
},

[yosifkit]

@HighPriest,

See https://docs.ghost.org/config#running-ghost-with-config-env-variables for setting environment variables for most any config value (linked from the Docker Hub docs):

For nested config options, separate with two underscores:

database__connection__host=mysql

So, in a docker run that'd be like the following:

$ docker run -e security__staffDeviceVerification=false -e logging__transports=stdout ... ghost

[HighPriest]

@yosifkit Doesn't work for me.
I've tried setting admin__url=whatever, but it seems to not be respected by the application. I have some other difficulties with configuration. As in, setting admin url through normal configuration results in "connection refused" on the proxy. So, I am going to play around with the config a little bit more before coming up with conclusions & recommendations.

[HighPriest]

Tested it again. And it seems to work. Although I am having difficulties verifying the actual configuration, because ghost config does not dump the configuration. ghost config admin.url or whatever other syntax I tried, does not show me nested keys.

I have ran into other issues, but I am going to keep the configuration through environment variables and only use VOLUME mount for content. It is actually quite a neat approach to setting up the config on many instances.

---

I am still against implicit definition of content path in the config file. It leads to seemingly random volumes being created by the containers orchestrator, to which there are no ties in configurations definitions.
With anonymous volumes, default behaviour in Docker is to purge them with container removal. So, their existence is a "maybe" and a better solution would be to emphasize in introduction docs, that explicit creation of a volume for content data is important for data persistence.