Using takserver-docker-5.3-RELEASE-4.zip as the Tak Server, the database gets lost when you bring the docker container down and back up.
To reproduce, setup using the latest Tak Server docker release, then run
docker-compose down
docker-compose up -d
After you're done, the database will be empty and completely reinitialized from scratch... but is missing the schema. If you look at Log1 at the bottom of this, you'll see that it's initializing a new database when the tak-server_db_1 container is started. My guess here is it never runs the SchemaManager.jar because Java isn't fully initialized -- I see the following in the log: takserver-setup-db.sh: 19: java: not found -- so the schema is never set up and all the tables are missing. TAK Server runs, but not very well.
After much experimentation, I discovered that the Tak Server's tak/db-utils/takserver-setup-db.sh script is overriding the default Postgresql database path -- instead of /var/lib/postgresql/data, it's forcing /var/lib/postgresql/15/data instead. Since the docker-compose.yml file is using the default path, the net effect is that when you bring the docker container down, you lose all the database data, and the database is reinitialized the next time the container is started.
Unfortunately, it's not enough to simply change docker-compose.yml to use - "db_data:/var/lib/postgresql/15/data:z" in the volumes section instead of the current db_data line because apparently the standard Postgres image expects the volume to be mounted at the original directory instead. I've tried a bunch of variations including modifying PGDATA and mounting two data volumes, one at the old and one at the new, but I couldn't get postgres to work this way at all -- that docker image really wants the DB to be at /var/lib/postgresql/data.
I tried takserver-docker-5.2-RELEASE-56.zip as well, with the same results. The 5.2 release 30 in the checksum files is no longer available on Tak.gov, so I suspect it had the same changes made. I also tried the takserver-docker-5.1-RELEASE-52.zip with the same results.
Finallly, I also noticed the scripts/setup.sh had three lines commented out which copied overrides to the TAK postgres setup and I tried the 5.3 release with that -- that one did work, but Tak Server came up with a bunch of suggestions on how to change the Postgres configuration. I'm going to continue working with that one for now, but I don't know if it needs further changing.
As a possibly related issue (although it might instead be a difference between my docker version - 27.3.1 build ce12230 or my docker-compose version 1.25.0), I noticed the scripts\cleanup.sh script uses "tak-server-db-data" as the volume name whereas on my setup it actually uses "tak-server_db_data". The same hyphen vs underscore issue also occurs in the docker image rm commands at the end of that script.
Finally, thank you so much for this project -- it resolved so many issues and questions we had getting started with TAK.
Log 1 - Showing the issue
- Output of
docker logs tak-server_db_1 after cleanup/setup and then a down/up cycle
/opt/tak/db-utils/configureInDocker.sh: line 5: ps: command not found
xargs: kill: No such file or directory
Initializing the database...
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/15/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
syncing data to disk ... ok
Success. You can now start the database server using:
/usr/lib/postgresql/15/bin/pg_ctl -D /var/lib/postgresql/15/data -l logfile start
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
Copying in our custom pg_hba.conf configuration file..
Starting the database...
waiting for server to start.... done
server started
Configuring the database...
TAK Server DB Setup
Checking if running in docker container...
Docker installation detected.
Debian installation detected.
Checking that postgresql 15 is installed...
exporting PGDATA variable
(Re)starting PostgreSQL service.
Installing TAKServer's version of PostgreSQL access-control policy.
Copied existing PostgreSQL access-control policy to /var/lib/postgresql/15/data/pg_hba.conf.backup-2024-12-10-23:15:34+00:00.
Docker db install
Installed TAKServer's PostgreSQL access-control policy to /var/lib/postgresql/15/data/pg_hba.conf.
Restarting PostgreSQL service.
Copied existing PostgreSQL configuration to /var/lib/postgresql/15/data/postgresql.conf.backup-2024-12-10-23:15:34+00:00.
Installed TAKServer's PostgreSQL configuration to /var/lib/postgresql/15/data/postgresql.conf.
Restarting PostgreSQL service.
Creating user "martiuser" ...
with SCRAM-SHA-256 password encryption
CREATE ROLE
Creating database cot
Database cot created.
Applying the TAKServer specific schema changes for the TAK Server version...
takserver-setup-db.sh: 19: java: not found
Database updated with SchemaManager.jar
Performing any TAK Server specific upgrades to the schema...
/opt/tak/db-utils/configureInDocker.sh: line 27: java: command not found
Restarting the database...
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
Using takserver-docker-5.3-RELEASE-4.zip as the Tak Server, the database gets lost when you bring the docker container down and back up.
To reproduce, setup using the latest Tak Server docker release, then run
After you're done, the database will be empty and completely reinitialized from scratch... but is missing the schema. If you look at Log1 at the bottom of this, you'll see that it's initializing a new database when the tak-server_db_1 container is started. My guess here is it never runs the SchemaManager.jar because Java isn't fully initialized -- I see the following in the log:
takserver-setup-db.sh: 19: java: not found-- so the schema is never set up and all the tables are missing. TAK Server runs, but not very well.After much experimentation, I discovered that the Tak Server's
tak/db-utils/takserver-setup-db.shscript is overriding the default Postgresql database path -- instead of/var/lib/postgresql/data, it's forcing/var/lib/postgresql/15/datainstead. Since thedocker-compose.ymlfile is using the default path, the net effect is that when you bring the docker container down, you lose all the database data, and the database is reinitialized the next time the container is started.Unfortunately, it's not enough to simply change
docker-compose.ymlto use- "db_data:/var/lib/postgresql/15/data:z"in the volumes section instead of the current db_data line because apparently the standard Postgres image expects the volume to be mounted at the original directory instead. I've tried a bunch of variations including modifying PGDATA and mounting two data volumes, one at the old and one at the new, but I couldn't get postgres to work this way at all -- that docker image really wants the DB to be at/var/lib/postgresql/data.I tried takserver-docker-5.2-RELEASE-56.zip as well, with the same results. The 5.2 release 30 in the checksum files is no longer available on Tak.gov, so I suspect it had the same changes made. I also tried the takserver-docker-5.1-RELEASE-52.zip with the same results.
Finallly, I also noticed the scripts/setup.sh had three lines commented out which copied overrides to the TAK postgres setup and I tried the 5.3 release with that -- that one did work, but Tak Server came up with a bunch of suggestions on how to change the Postgres configuration. I'm going to continue working with that one for now, but I don't know if it needs further changing.
As a possibly related issue (although it might instead be a difference between my docker version - 27.3.1 build ce12230 or my docker-compose version 1.25.0), I noticed the
scripts\cleanup.shscript uses "tak-server-db-data" as the volume name whereas on my setup it actually uses "tak-server_db_data". The same hyphen vs underscore issue also occurs in thedocker image rmcommands at the end of that script.Finally, thank you so much for this project -- it resolved so many issues and questions we had getting started with TAK.
Log 1 - Showing the issue
docker logs tak-server_db_1after cleanup/setup and then a down/up cycle