diff --git a/Docker.md b/Docker.md index 3239ef3..7fd7dd4 100644 --- a/Docker.md +++ b/Docker.md @@ -2,7 +2,7 @@ ## Docker Support in LSB -The core team of LSB does not use Docker in their workflows, and as such can't properly maintain a docker setup as a first-class citizen in the LSB project. Over time, people would pop in with functioning Docker/compose files, which would then decay after they left. This article is a compromise for those that want to use docker as part of their workflow, where we'll try our best to keep some working example files here that anyone can drop into their LSB server project and run. +This article is for those that want to use Docker as part of their workflow, to share some working example files here that anyone can drop into their LSB server project and run. See [the README](https://github.com/LandSandBoat/server/tree/base/docker) for official Docker support. ## No GitHub Issues + Support @@ -20,7 +20,6 @@ The goal of the provided files was as follows: The caveats of the provided examples are: -- Performance may suffer since the different pieces of the server are not being decoupled. - The configuration provided are defaults for local development and **should not be used for a production deployment**. ## Docker Architecture @@ -46,187 +45,75 @@ flowchart LR I[Internet] --> RP ``` -## update_db_then_launch.sh -```bash -#!/bin/bash - -while ! mysql --host=$XI_NETWORK_SQL_HOST --port=$XI_NETWORK_SQL_PORT --user=$XI_NETWORK_SQL_LOGIN --password=$XI_NETWORK_SQL_PASSWORD $XI_NETWORK_SQL_DATABASE -e "SELECT 1 FROM zone_weather LIMIT 1"; do - sleep 5 -done -sleep 5 - -# Update databse -echo "updating database" -python3 ./tools/dbtool.py update -sleep 5 - -# Update zone_settings to host IP -mysql --host=$XI_NETWORK_SQL_HOST --port=$XI_NETWORK_SQL_PORT --user=$XI_NETWORK_SQL_LOGIN --password=$XI_NETWORK_SQL_PASSWORD $XI_NETWORK_SQL_DATABASE -e "UPDATE zone_settings set zoneip='$XI_NETWORK_ZONE_IP'" - -# Start servers -echo "starting xi_connect" -nohup ./xi_connect & -sleep 5 - -echo "starting xi_search" -nohup ./xi_search & - -sleep 5 -echo "starting xi_map" -./xi_map - -sleep 5 -echo "starting xi_world" -./xi_world -``` - -## Dockerfile - -```docker -FROM ubuntu:24.04 - -RUN apt clean - -# Avoid any UI since we don't have one -ENV DEBIAN_FRONTEND=noninteractive - -# Working directory will be /server meaning that the contents of server will exist in /server -WORKDIR /server - -# Some dependencies are pulled from deadsnakes -RUN apt update && apt install -y wget curl software-properties-common -RUN add-apt-repository ppa:deadsnakes/ppa - -# Need mariadb as per-requirements, doesn't come pre-packaged I don't think -RUN apt-get install -y libmariadb3 libmariadb-dev mariadb-server - -RUN apt-get update -RUN apt-get upgrade -y - -RUN apt install -y python3.12 python3.12-dev python3-pip - -RUN python3 --version - -# Update and install all requirements as well as some useful tools such as net-tools and nano -RUN apt install -y net-tools nano git cmake make libluajit-5.1-dev libzmq3-dev libssl-dev zlib1g-dev luarocks binutils-dev - -# Copy everything from the host machine server folder to /server -ADD . /server - -# New python requires us to either create a virtual ENV or -# set --break-system-packages at our own risk. Since this is -# a one-off docker context, we don't care if it mucks with -# system packages as long as it all works in the end. -RUN pip3 install --upgrade --break-system-packages -r ./tools/requirements.txt - -# Configure and build -RUN mkdir docker_build && cd docker_build && cmake .. && make -j $(nproc) && cd .. && rm -r /server/docker_build - -# Ensure we can run the db update -RUN chmod +x ./tools/dbtool.py - -# Ensure we can run the startup script -RUN chmod +x ./update_db_then_launch.sh - -# Startup the server when the container starts -ENTRYPOINT ./update_db_then_launch.sh -``` - ## docker-compose.yml ```yml -version: '3.1' +x-dbcreds: &dbcreds + # MARIADB_ROOT_PASSWORD required if setting up fresh database. + # Or generate a random root password and print it to build log: + # MARIADB_RANDOM_ROOT_PASSWORD: true + MARIADB_ROOT_PASSWORD: 'root' + MARIADB_DATABASE: xidb + MARIADB_USER: xiadmin + MARIADB_PASSWORD: 'password' + +x-common: &common + image: ghcr.io/landsandboat/server:latest + environment: + <<: *dbcreds + XI_NETWORK_HTTP_HOST: 0.0.0.0 + XI_NETWORK_ZMQ_IP: world + XI_NETWORK_SQL_HOST: database + XI_NETWORK_ZONE_IP: 192.168.11.11 # UPDATE THIS TO THE LAN IP OF THE HOST MACHINE + # XI_{file}_{setting}: value + volumes: + - losmeshes:/server/losmeshes + - navmeshes:/server/navmeshes + # - ./config.yaml:/server/tools/config.yaml + # - ./map.lua:/server/settings/map.lua + # - ./modules:/server/modules + networks: + - lsb + labels: + - "traefik.enable=true" + - "traefik.udp.routers.game.service=svc_game" + - "traefik.udp.services.svc_game.loadbalancer.server.port=54230" + - "traefik.tcp.routers.connect.rule=HostSNI(`*`)" + - "traefik.tcp.routers.connect.entrypoints=connect" + - "traefik.tcp.routers.connect.service=svc_connect" + - "traefik.tcp.services.svc_connect.loadbalancer.server.port=54230" + - "traefik.tcp.routers.connect1.rule=HostSNI(`*`)" + - "traefik.tcp.routers.connect1.entrypoints=connect1" + - "traefik.tcp.routers.connect1.service=svc_connect1" + - "traefik.tcp.services.svc_connect1.loadbalancer.server.port=54231" + - "traefik.tcp.routers.connect2.rule=HostSNI(`*`)" + - "traefik.tcp.routers.connect2.entrypoints=connect2" + - "traefik.tcp.routers.connect2.service=svc_connect2" + - "traefik.tcp.services.svc_connect2.loadbalancer.server.port=54001" + - "traefik.tcp.routers.search.rule=HostSNI(`*`)" + - "traefik.tcp.routers.search.entrypoints=search" + - "traefik.tcp.routers.search.service=svc_search" + - "traefik.tcp.services.svc_search.loadbalancer.server.port=54002" -services: - # The DB service - db: - image: mariadb - restart: always - environment: - MYSQL_USER: xiuser - MYSQL_PASSWORD: xipassword - MYSQL_ROOT_PASSWORD: root - MYSQL_DATABASE: xidb - networks: - - lsb - # Run all the .sql files in the /sql directory to initalize the DB. This only hapens the first time this service is started and will not handle additions/modifications - volumes: - - ./sql:/docker-entrypoint-initdb.d - - ~/lsb_server/data:/var/lib/mysql - ports: - - "3306:3306" +services: # Ease of access tool for the DB, you can type in localhost:8080 to get a web interface to the DB. You can log in with root:wheel db_admin_portal: image: adminer restart: always depends_on: - - db + - database networks: - lsb ports: - 8080:8080 - # The server service - game: - # Build whatever is in the Dockerfile in the server root folder - build: . - environment: - XI_NETWORK_SQL_HOST: db - XI_NETWORK_SQL_PORT: 3306 - XI_NETWORK_SQL_LOGIN: xiuser - XI_NETWORK_SQL_PASSWORD: xipassword - XI_NETWORK_SQL_DATABASE: xidb - XI_NETWORK_LOGIN_DATA_IP: 0.0.0.0 - XI_NETWORK_LOGIN_DATA_PORT: 54230 - XI_NETWORK_LOGIN_VIEW_IP: 0.0.0.0 - XI_NETWORK_LOGIN_VIEW_PORT: 54001 - XI_NETWORK_LOGIN_AUTH_IP: 0.0.0.0 - XI_NETWORK_LOGIN_AUTH_PORT: 54231 - XI_NETWORK_ZMQ_IP: 0.0.0.0 - XI_NETWORK_ZMQ_PORT: 54003 - # UPDATE THIS TO THE LAN IP OF THE HOST MACHINE - XI_NETWORK_ZONE_IP: 192.168.50.248 - XI_NETWORK_MAP_PORT: 54230 - XI_NETWORK_SEARCH_PORT: 54002 - depends_on: - - db - - traefik - networks: - - lsb - labels: - - "traefik.enable=true" - - - "traefik.udp.routers.game.service=svc_game" - - "traefik.udp.services.svc_game.loadbalancer.server.port=54230" - - - "traefik.tcp.routers.connect.rule=HostSNI(`*`)" - - "traefik.tcp.routers.connect.entrypoints=connect" - - "traefik.tcp.routers.connect.service=svc_connect" - - "traefik.tcp.services.svc_connect.loadbalancer.server.port=54230" - - - "traefik.tcp.routers.connect1.rule=HostSNI(`*`)" - - "traefik.tcp.routers.connect1.entrypoints=connect1" - - "traefik.tcp.routers.connect1.service=svc_connect1" - - "traefik.tcp.services.svc_connect1.loadbalancer.server.port=54231" - - - "traefik.tcp.routers.connect2.rule=HostSNI(`*`)" - - "traefik.tcp.routers.connect2.entrypoints=connect2" - - "traefik.tcp.routers.connect2.service=svc_connect2" - - "traefik.tcp.services.svc_connect2.loadbalancer.server.port=54001" - - - "traefik.tcp.routers.search.rule=HostSNI(`*`)" - - "traefik.tcp.routers.search.entrypoints=search" - - "traefik.tcp.routers.search.service=svc_search" - - "traefik.tcp.services.svc_search.loadbalancer.server.port=54002" - traefik: image: traefik restart: unless-stopped environment: TRAEFIK_API_DASHBOARD: false TRAEFIK_API_INSECURE: true - TRAEFIK_ENTRYPOINTS_GAME: true TRAEFIK_ENTRYPOINTS_GAME_ADDRESS: ":54230/udp" TRAEFIK_ENTRYPOINTS_CONNECT: true @@ -237,12 +124,10 @@ services: TRAEFIK_ENTRYPOINTS_CONNECT2_ADDRESS: ":54001/tcp" TRAEFIK_ENTRYPOINTS_SEARCH: true TRAEFIK_ENTRYPOINTS_SEARCH_ADDRESS: ":54002/tcp" - TRAEFIK_PROVIDERS_DOCKER: true TRAEFIK_PROVIDERS_DOCKER_WATCH: true TRAEFIK_PROVIDERS_DOCKER_NETWORK: "web" TRAEFIK_PROVIDERS_DOCKER_EXPOSEDBYDEFAULT: false - ports: - "54231:54231/tcp" - "54230:54230/tcp" @@ -255,9 +140,98 @@ services: - web - lsb + database: + image: mariadb:lts + restart: always + command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_general_ci'] + environment: + <<: *dbcreds + volumes: + - database:/var/lib/mysql + networks: + - lsb + healthcheck: + test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"] + start_period: 10s + interval: 10s + timeout: 5s + retries: 3 + + database-update: + <<: *common + command: ["python", "/server/tools/dbtool.py", "update"] + depends_on: + database: + condition: service_healthy + traefik: + condition: service_healthy + + connect: + <<: *common + command: ["/server/xi_connect"] + restart: unless-stopped + ports: + - "54001:54001" + - "54230:54230" + - "54231:54231" + depends_on: + database: + condition: service_healthy + restart: true + database-update: + condition: service_completed_successfully + + search: + <<: *common + command: ["/server/xi_search"] + restart: unless-stopped + ports: + - "54002:54002" + depends_on: + database: + condition: service_healthy + restart: true + database-update: + condition: service_completed_successfully + + world: + <<: *common + command: ["/server/xi_world"] + restart: unless-stopped + ports: + - "8088:8088" + depends_on: + database: + condition: service_healthy + restart: true + database-update: + condition: service_completed_successfully + + map: + <<: *common + command: ["/server/xi_map"] + restart: unless-stopped + ports: + - "54230:54230/udp" + depends_on: + database: + condition: service_healthy + restart: true + database-update: + condition: service_completed_successfully + world: + condition: service_started + networks: lsb: external: false web: external: true + +volumes: + database: + losmeshes: + external: true + navmeshes: + external: true ``` diff --git a/Quick-Start-Guide.md b/Quick-Start-Guide.md index dc9834f..8c59dad 100644 --- a/Quick-Start-Guide.md +++ b/Quick-Start-Guide.md @@ -1,4 +1,35 @@ -# Main Supported Platforms +# Quick Start + +## Docker + +The fastest way to get started is through [Docker](https://docs.docker.com/). + +Checking out the repository with Git is not required for this method. + +* Create a file named `docker-compose.yml` using the [example in the README](https://github.com/LandSandBoat/server/tree/base/docker#via-docker-compose) (consider changing the passwords at the top). + * You can put this in an empty folder. + +* Run the following command to install meshes: + +```sh +docker run --rm -v losmeshes:/losmeshes -v navmeshes:/navmeshes ghcr.io/landsandboat/ximeshes:latest +``` + +* Start the server: + +```sh +docker compose up --detach +``` + +### Updating + +```sh +docker compose pull && docker compose up -d +``` + +## Building from source + +### Main Supported Platforms **NOTE:** While it's possible to achieve the steps in this guide using different tools and versions of the software we list, we _cannot recommend enough_ following this guide to the letter and make sure you have everything working before you stray from the well-tested path. Using software parts from a pre-made stack like WAMP, the built-in Python that ships with Strawberry Perl, etc. is NOT suitable for the first time you're getting set up. Please follow the instructions to the letter and only start swapping pieces out once you have a working server. @@ -9,7 +40,9 @@ _(click to expand sections)_
Windows 10/11 -## To Install +#### Windows + +##### Install (Windows) * Install [Git for Windows](https://gitforwindows.org/). * The latest version is fine, accept defaults, change default text editor if desired. @@ -85,7 +118,7 @@ o------------------------------------------o * You should eventually see `Build All succeeded.`. * Congratulations, you've built the server! You can now go onto [Next Steps](#next-steps). -## To Update +##### Update (Windows) * **Take down all of your server processes!** * Open a PowerShell window and navigate to your `server` directory. @@ -123,14 +156,16 @@ py -3 dbtool.py update
Linux (Ubuntu 24.04) -## To Install +#### Linux (Ubuntu) + +##### Install (Ubuntu) ```txt NOTE: We try to keep up to date with whatever the latest LTS release of Ubuntu is (Ubuntu 24.04). We run all of our CI builds on this release. We can't guarantee that older LTS versions will work. When in doubt: update! ``` * Run these steps to use Mariadb's community provided ("CS" instructions) .deb packages through apt: - * https://mariadb.com/docs/server/connect/programming-languages/c/install/#CS_Package_Repository + * * Use your package manager to install the following packages or their equivalents: ```sh @@ -152,7 +187,7 @@ cp server/settings/default/* server/settings sudo mysql_secure_installation ``` -* Type the following to create a database user with the login _**xi**_ and password _**password**_, and an empty database called _**xidb**_. NOTE: You _SHOULD_ change **ALL THREE OF THESE** to improve security: +* Type the following to create a database user with the login _**xi**_ and password _**password**_, and an empty database called _**xidb**_. NOTE: You _SHOULD_ change **ALL THREE OF THESE** to improve security: ```sh sudo mysql -u root -p -e "CREATE USER 'xi'@'localhost' IDENTIFIED BY 'password';CREATE DATABASE xidb CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;USE xidb;GRANT ALL PRIVILEGES ON xidb.* TO 'xi'@'localhost';" @@ -182,7 +217,7 @@ python3 dbtool.py * Congratulations, you've built and set up the server! You can now go onto [Next Steps](#next-steps). -## To Update +##### Update (Ubuntu) * **Take down all of your server processes!** * Open the `server` directory in a terminal. @@ -220,7 +255,77 @@ python3 dbtool.py update ----- -# Experimental Platforms +
+ Docker + +#### Build with Docker + +* First you need to clone the repo: + +```sh +git clone https://github.com/LandSandBoat/server.git +``` + +* You don't need the submodules, and can use the same command as above to store them in a volume: + +```sh +docker run --rm -v losmeshes:/losmeshes -v navmeshes:/navmeshes ghcr.io/landsandboat/ximeshes:latest +``` + +* Copy [dev.docker-compose.yml](https://github.com/LandSandBoat/server/tree/base/dev.docker-compose.yml) to `docker-compose.yml`: + +```sh +cp dev.docker-compose.yml docker-compose.yml +``` + +* Build the image: + * You can use `--build clang_tidy` instead of `--build build`. + * It will take longer but you'll get clang-tidy results in `log/clang_tidy_summary.md`. + * Both `build` and `clang_tidy` build the `landsandboat/server:testing` image used for the other services. + +```sh +docker compose up -d --build build +``` + +* Start the database: + * By default, there's no volume for the database, so any changes are deleted when you remove the containers. You can add a volume to your `docker-compose.yml` if you want the changes to persist. + +```sh +docker compose up -d database +``` + +* Update the database: + +```sh +docker compose up -d setup_database +``` + +* Run sanity checks (the output will be in `sanity_checks_summary.md`): + +```sh +docker compose up -d sanity_checks +``` + +* Run xi_test (the output will be in `log/tests.ctrf.json`): + +```sh +docker compose up -d test +``` + +* Run the server: + * This will start the database, world, connect, and search services if they haven't been started already. + +```sh +docker compose up -d map +``` + +Those are the basics. You can edit your `docker-compose.yml` to fit your needs and tastes. See [the README](https://github.com/LandSandBoat/server/tree/base/docker) for more. + +
+ +----- + +### Experimental Platforms **NOTE:** These platforms should work, but are not actively maintained or used by the development team. The development team (especially in the case of OSX) might not have the hardware or expertise to be able to help you debug problems on these platforms. Use at your own risk. Good luck! @@ -229,9 +334,9 @@ _(click to expand sections)_ -----
- OSX + macOS -## To Install +#### macOS * Get dependencies from brew: @@ -241,7 +346,7 @@ brew install git pkg-config autoconf make cmake gcc openssl mariadb zeromq zmqpp * The version of LuaJIT that you can get through brew is old. You can build and install LuaJIT for your system with: -``` +```sh git clone https://github.com/LuaJIT/LuaJIT.git cd LuaJIT sudo make install MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion) -j $(sysctl -n hw.physicalcpu) @@ -250,7 +355,7 @@ sudo ln -sf luajit-2.1.0-beta3 /usr/local/bin/luajit * Download and build the server binaries: -``` +```sh git clone --recursive https://github.com/LandSandBoat/server.git mkdir build cd build @@ -263,14 +368,16 @@ From here, the instructions are the same as the Linux builds. Good luck! NOTE: You may have problems with missing symbols from LuaJIT. This happens if the build system picks up LuaJIT's headers instead of our internal (and expected) ones. We discovered this in [this discussion](https://github.com/LandSandBoat/server/discussions/1015). In your CMake configuration, you should see this: -``` + +```sh -- LuaJIT_FOUND: TRUE -- LuaJIT_LIBRARY: /usr/local/lib/libluajit-5.1.dylib -- LuaJIT_INCLUDE_DIR: /Users/runner/work/server/server/ext/lua/include ``` If the `LuaJIT_INCLUDE_DIR` is pointing somewhere other than `/server/server/ext/lua/include`, you can change it during CMake configuration by using: -``` + +```sh cmake .. -DLuaJIT_INCLUDE_DIR=/server/ext/lua/include ``` @@ -281,6 +388,8 @@ cmake .. -DLuaJIT_INCLUDE_DIR=/server/ext/lua/include
Linux (through WSL) +#### WSL + All of the instructions for Linux should be valid for WSL. There are additional points covered in the [Working with WSL](Working-with-WSL) article.
@@ -290,6 +399,8 @@ All of the instructions for Linux should be valid for WSL. There are additional
Linux (Arch) +#### Linux (Arch) + Some users have had success building and running on Arch. We can't and won't support Arch as main platform. Good luck! ```sh @@ -309,19 +420,21 @@ sudo systemctl start mariadb
Linux (Raspberry Pi) +#### Linux (Raspberry Pi) + Build instructions should be the same or similar as a regular Linux build. The build process may take a long time, but running the game doesn't take much computing power. -#### Power +##### Power Raspberry Pis require at least a 2.5amp power supply to run at full power. If you are getting a little yellow lightning bolt in the top right of your display you have hit the limit of your current power supply. If this happens you may not be able to take full advantage of your CPU's power and may lose connectivity to Bluetooth or USB devices. Should you hit either of these 2 limitations it will take considerably longer for the build process to finish, if it finishes at all! -#### LuaJIT +##### LuaJIT Depending on your distro, the LuaJIT that comes through the package manager may not have required fixes for ARM platforms included with it. It's recommended you follow the steps in the OSX build guide to build and use the latest LuaJIT. -#### RAM +##### RAM Each server process startup can be quite resource intensive for both CPU and RAM. Older Raspberry Pis don't have much RAM, so you may need to start up each of the server processes one-by-one to ensure that they start and run correctly. @@ -332,54 +445,61 @@ Each server process startup can be quite resource intensive for both CPU and RAM
Linux (Gentoo OpenRC) -Ensure your system is up to date: +#### Linux (Gentoo OpenRC) + +Ensure your system is up to date:\ + ```sh sudo emerge --sync && emerge -avuDU @world ``` -Emerge the following packages and their dependencies: + +Emerge the following packages and their dependencies: + ```sh sudo emerge -a dev-db/mariadb dev-lang/luajit dev-vcs/git net-libs/zeromq ``` + Clone the repo in your folder of choice, then copy the settings files: + ```sh cd ~/ && mkdir git && cd ~/git git clone --recursive https://github.com/LandSandBoat/server.git cp server/settings/default/* server/settings ``` + MariaDB will need to be configured and the database initialized before the service can be started. If you have issues, or are using Systemd instead of OpenRC, refer to the [Gentoo Wiki](https://wiki.gentoo.org/wiki/MariaDB). + ```sh sudo emerge --config dev-db/mariadb sudo rc-update add mysql default sudo rc-service mysql start ``` + In order to use dbtool for managing your database, additional packages are required, one of which is not in the main Gentoo repository. This is a problem on Gentoo as installing with pip instead of portage can break your system. Thankfully, with an overlay we can get what we need (ensure you have already installed and configured [eselect-repository](https://wiki.gentoo.org/wiki/Eselect/Repository)): + ```sh sudo eselect repository add claytabase git https://github.com/claybie/claytabase.git sudo emaint sync -r claytabase ``` + Now we can emerge all the necessary packages for dbtool: + ```sh sudo emerge -a dev-python/black dev-python/colorama dev-python/GitPython dev-python/mariadb dev-python/pylint dev-python/pyyaml dev-python/pyzmq dev-python/regex ``` + Additionally, you will also need to emerge the below packages if you wish to use [pydarkstar](https://github.com/AdamGagorik/pydarkstar) as an automated auction house: + ```sh sudo emerge -a dev-python/beautifulsoup4 dev-python/sqlalchemy ``` -The process for securing the MariaDB installation, creating the SQL database, building the project with make, populating the database using dbtool and performing future updates is the same as on Ubuntu. It can be referenced above from the *Linux (Debian/Ubuntu 22.04)* section. -
- ------ - -
- Docker - -The core team of LSB does not use Docker in their workflows, and as such can't properly maintain a Docker setup as a first-class citizen. There is an unofficial Docker guide [here](Docker). +The process for securing the MariaDB installation, creating the SQL database, building the project with make, populating the database using dbtool and performing future updates is the same as on Ubuntu. It can be referenced above from the *Linux (Debian/Ubuntu 22.04)* section.
----- -# Next Steps +## Next Steps * The most basic way to start your server is to launch the newly-built `xi_*` executables, found in your repo root: * `xi_connect.exe`