TUI menu, configurable JDBC connections, drivers, docs + CI

.github/workflows/ci.yml

@@ -0,0 +1,27 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '21'
+          cache: gradle
+
+      # ubuntu-latest ships a working Docker daemon, so the Testcontainers integration
+      # tests (PostgreSQL -> MariaDB) run here; the rest of the suite needs no services.
+      - name: Build and test
+        run: ./gradlew build --no-daemon --stacktrace

CHANGELOG.md

@@ -0,0 +1,55 @@
+# Changelog
+
+All notable changes to DataLinq are documented here.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and the project aims to adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+First functional cut: a cross-vendor JDBC data-migration tool with a TamboUI front-end.
+Nothing is released to a registry yet; the version is `0.1.0-SNAPSHOT`.
+
+### Added
+
+- **Migration engine** - folder-scanning operation discovery (`sql/NN_Name/`), three
+  operation types (ETL row copy, SCRIPT on the target, custom HANDLER), each run in a single
+  target transaction (commit on success, rollback on error), **dry-run by default**.
+- **Handlers** via `MigrationHandler` + ServiceLoader (no string reflection, GraalVM-friendly),
+  with helpers for master/detail splits and generated-key FKs, plus coercing typed `Row`
+  accessors (`getLong/getInt/getBigDecimal/getBool`).
+- **Parallel batch runner** on virtual threads, bounded by `max-parallel`.
+- **`application.yml` config** - named multi-datasource pool (any datasource can be a source or
+  target), `defaults.source/target`, per-operation `source=`/`target=` overrides, and options
+  (`batch-size`, `dry-run-default`, `language`, `max-parallel`, `mask-password`, `sql-dir`).
+- **Structured connections** - datasources configured by `type` + `host`/`port`/`database`
+  (sqlserver / mariadb / postgresql), with a `custom` type that keeps a hand-written URL.
+- **JDBC drivers** - SQL Server, MariaDB/MySQL and PostgreSQL bundled; others downloadable from
+  Maven Central via `datalinq driver <name>` (oracle / mysql / h2 / sqlite) into
+  `~/.datalinq/drivers/`. Externally-loaded drivers are registered through a `DriverShim` so the
+  JDK `DriverManager` will use them.
+- **TamboUI TUI** (MVC; the controller is TUI/DB-free and unit-tested) with a Migrations menu
+  (number shortcuts, mouse, run/dry-run/run-all), a **DB Connection** screen (two-pane datasource
+  list / structured edit fields, test + save, default source/target, derived-URL preview), a
+  **Settings** screen (options + a folder picker for the sql dir, with live menu refresh), and
+  About / destructive-confirm dialogs. CJK-aware column alignment throughout.
+- **CLI** - `tui` (default), `init`, `list`, `config`, `run <index> [--execute]`, `driver`,
+  `i18n`, `logo`.
+- **Distribution** - single droppable Shadow fat-jar (`datalinq.jar`) with default resources
+  baked in, a jbang alias, and `datalinq init` to materialise editable defaults.
+- **i18n** (English / Korean) from external `messages_<lang>.properties` overlaying the bundled
+  defaults, and an external brand logo.
+
+### Fixed
+
+- Driver re-download failed on Windows with a cryptic file-lock error: external drivers are now
+  loaded only for the commands that open connections (`tui`/`run`), so `driver <name>` can refresh
+  an already-downloaded jar. Downloads also report HTTP status clearly and stage to a `.part` file
+  before an atomic move (no truncated jars).
+- Crash (`NoClassDefFoundError`) when pressing Esc on the main screen.
+- DB Connection screen: column misalignment with CJK labels, and editable fields that could not be
+  reached / edited (now a two-pane Left/Right navigation model).
+
+### Notes
+
+- `application.yml` holds credentials and is gitignored; copy `application.example.yml`.
+- Targets Java 21 bytecode (Gradle toolchain), runs on any JDK 21+.

README.md

@@ -4,17 +4,20 @@ A TUI-driven data migration tool (built with [TamboUI](https://github.com/tambou
 The **base menus are fixed in code**; the **migration menus are discovered by scanning the
 `sql/` folder** - drop a folder, get a menu.
 
-Source = **MS SQL Server**, Target = **MariaDB** (cross-vendor, two connections).
+Moves rows **between any two JDBC databases** (cross-vendor, source + target). SQL Server,
+MariaDB/MySQL and PostgreSQL drivers are **bundled**; others (Oracle, H2, SQLite, ...) are a
+one-line `driver` download away. Connections are configured by **DB type + host/port/database**
+(or a raw URL), in `application.yml` or live from the DB Connection screen.
 
-> A devslab asset. Licensed under Apache-2.0.
+> By **DevsLab Co., Ltd.** (주식회사 데브스랩) · https://devslab.kr · Apache-2.0
 
 ## Add a migration = drop a folder
 
 ```
 sql/
 ├── 01_Approval_Lines/       # ETL: source.sql (aliased SELECT) -> target table
 │   ├── source.sql
-│   └── operation.properties     ->  type=etl, target=approval_lines
+│   └── operation.properties     ->  type=etl, table=approval_lines
 ├── 03_Reset_Base_Data/      # SCRIPT: run .sql on the TARGET (reset/delete)
 │   ├── 01_reset.sql
 │   └── operation.properties     ->  type=script, destructive=true
@@ -30,7 +33,7 @@ sql/
 
 | type | folder shape | what it does | code? |
 |------|--------------|--------------|-------|
-| **etl** | `source.sql` + `target=<table>` | read source, auto-INSERT into target. The SELECT's **column aliases = target columns**, so no INSERT is written. | none |
+| **etl** | `source.sql` + `table=<table>` | read source, auto-INSERT into target. The SELECT's **column aliases = target columns**, so no INSERT is written. | none |
 | **script** | one or more `.sql` | run them against the **target** (resets, deletes). | none |
 | **handler** | `source.sql` + `handler=<name>` | a `MigrationHandler` (transforms / master-detail / generated-key FKs), discovered by ServiceLoader. | ~20 lines |
 
@@ -54,37 +57,89 @@ so a GraalVM native image stays possible.
 
 ## Configuration (`application.yml`)
 
+Multiple **named datasources** - any can be a source or a target. Operations pick by name,
+falling back to `defaults`:
+
 ```yaml
-datasource:
-  source: { url: jdbc:sqlserver://..., username: sa,   password: "" }
-  target: { url: jdbc:mariadb://...,   username: root, password: "" }
+datasources:
+  # Structured types build the JDBC URL from host/port/database:
+  legacy-erp:                      # source
+    type: sqlserver                # sqlserver | mariadb | postgresql
+    host: SRC_HOST
+    port: 1433
+    database: SRC_DB
+    username: sa
+    password: ""
+  new-core:                        # target (mariadb also connects to MySQL servers)
+    type: mariadb
+    host: TGT_HOST
+    port: 3306
+    database: TGT_DB
+    username: root
+    password: ""
+  # custom type keeps a hand-written URL verbatim (use this for downloaded drivers):
+  # warehouse:
+  #   type: custom
+  #   url: jdbc:oracle:thin:@HOST:1521:ORCL
+  #   username: app
+  #   password: ""
+defaults:
+  source: legacy-erp               # used when an operation does not set source=
+  target: new-core                 # used when an operation does not set target=
 options:
   batch-size: 1000
   dry-run-default: true
+  language: en                     # en | ko  (blank = system locale)
+  max-parallel: 4                  # max operations run concurrently (bounds DB connections)
+  mask-password: true              # DB Connection screen: mask passwords (false = show plain)
+  # sql-dir: /path/to/sql          # external migration folder (blank = ./sql)
 ```
 
-Copy `application.example.yml` -> `application.yml` (gitignored). It can also be edited from
-inside the app (DB Connection screen) and saved.
+An operation can override per run with `source=<name>` / `target=<name>` in its
+operation.properties. Copy `application.example.yml` -> `application.yml` (gitignored); it can
+also be edited from inside the app (DB Connection screen) and saved.
+
+## Database drivers
+
+- **Bundled** (always work): SQL Server, MariaDB / MySQL, PostgreSQL.
+- **Downloadable** into `~/.datalinq/drivers/` from Maven Central, loaded on the next launch:
+
+  ```bash
+  datalinq driver               # list what is downloadable
+  datalinq driver oracle        # fetch one (oracle | mysql | h2 | sqlite | postgresql)
+  ```
+
+  You can also just drop any JDBC driver `.jar` into that folder by hand. Externally-loaded
+  drivers are registered through a `DriverShim` (the JDK's `DriverManager` ignores drivers from
+  a foreign class loader otherwise). Use them with the **custom** type + a hand-written URL.
 
 ## Safety
 
 - **dry-run by default** - the target transaction is rolled back, nothing is written.
 - `destructive=true` operations require an explicit confirmation before running.
 - Each operation runs in **one target transaction**: commit on success, rollback on any error.
 
-## Run (CLI)
+## Run
+
+The default command is the **TUI**; everything is also scriptable from the CLI.
 
 ```bash
-cp application.example.yml application.yml      # fill in MSSQL + MariaDB
-./gradlew run --args="list"                     # list operations
-./gradlew run --args="config"                   # show resolved config (passwords masked)
-./gradlew run --args="run 0"                    # dry-run operation #0
-./gradlew run --args="run 0 --execute"          # actually write
-# ./gradlew run                                 # TUI menu (wired next)
+./gradlew shadowJar                             # build the self-contained jar
+java -jar build/libs/datalinq.jar               # TUI menu (Migrations / DB Connection / Settings / About)
+
+# or via the CLI:
+java -jar build/libs/datalinq.jar init          # write editable defaults (i18n/, branding/, example config, sql/)
+java -jar build/libs/datalinq.jar list          # list discovered operations
+java -jar build/libs/datalinq.jar config        # show resolved config (passwords masked)
+java -jar build/libs/datalinq.jar run 0         # dry-run operation #0
+java -jar build/libs/datalinq.jar run 0 --execute   # actually write
 ```
 
+The jar is a single droppable artifact (Shadow fat-jar, with a [jbang](https://www.jbang.dev/)
+alias in `jbang-catalog.json`). During development `./gradlew run --args="..."` works too.
+
 ## Status
 
-Engine (scanner / ETL / script / handler / transactions / dry-run) + `application.yml` config
-+ CLI are in place and verified. The TamboUI menu front-end (Settings / DB Connection / About +
-migrations) is the next layer - it calls the same `MigrationEngine`.
+Engine (scanner / ETL / script / handler / transactions / dry-run), `application.yml` config,
+bundled + downloadable JDBC drivers, the CLI, and the **TamboUI TUI** (Migrations, DB Connection,
+Settings, About - all calling the same `MigrationEngine`) are in place and verified.

application.example.yml

@@ -1,16 +1,38 @@
 # Copy to application.yml and fill in. (application.yml is gitignored.)
-# You can also edit these from inside the app (DB Connection screen) and save.
+# You can also edit datasources from inside the app (DB Connection screen) and save.
 
-datasource:
-  source:                 # read side - MS SQL Server
-    url: jdbc:sqlserver://SRC_HOST:1433;databaseName=SRC_DB;encrypt=true;trustServerCertificate=true
+datasources:
+  # Each is a named connection; any can act as a source or a target (chosen per operation).
+  # Structured types (sqlserver / mariadb) build the JDBC URL from host/port/database.
+  legacy-erp:                 # MS SQL Server source
+    type: sqlserver
+    host: SRC_HOST
+    port: 1433
+    database: SRC_DB
     username: sa
     password: ""
-  target:                 # write side - MariaDB
-    url: jdbc:mariadb://TGT_HOST:3306/TGT_DB
+  new-core:                   # MariaDB (also connects to MySQL servers) target
+    type: mariadb
+    host: TGT_HOST
+    port: 3306
+    database: TGT_DB
     username: root
     password: ""
+  # custom type keeps a hand-written URL verbatim (legacy 'url:' without a type also works):
+  # other:
+  #   type: custom
+  #   url: jdbc:postgresql://HOST:5432/DB
+  #   username: app
+  #   password: ""
+
+defaults:
+  source: legacy-erp          # used when an operation does not set source=
+  target: new-core            # used when an operation does not set target=
 
 options:
   batch-size: 1000
   dry-run-default: true
+  language: en                # en | ko  (blank = system locale)
+  max-parallel: 4             # max operations run concurrently (virtual threads; bounds DB connections)
+  mask-password: true         # DB Connection screen: mask password values (false = show in plain)
+  # sql-dir: /path/to/sql     # external migration folder (blank = ./sql)

build.gradle.kts

@@ -1,5 +1,6 @@
 plugins {
     application
+    id("com.gradleup.shadow") version "9.4.2"
 }
 
 group = "kr.devslab"
@@ -14,11 +15,6 @@ repositories {
     }
 }
 
-java {
-    sourceCompatibility = JavaVersion.VERSION_21
-    targetCompatibility = JavaVersion.VERSION_21
-}
-
 dependencies {
     // TUI front-end (engine itself uses none of this)
     implementation("dev.tamboui:tamboui-toolkit:0.4.0-SNAPSHOT")
@@ -27,16 +23,98 @@ dependencies {
     // application.yml config (loaded as a Map -> native-image friendly)
     implementation("org.yaml:snakeyaml:2.3")
 
-    // JDBC drivers: source = MS SQL Server, target = MariaDB
+    // Bundled JDBC drivers (structured types). The MariaDB driver also connects to MySQL servers.
+    // Other databases work via the Custom type + a driver dropped in / downloaded to the drivers dir.
     implementation("com.microsoft.sqlserver:mssql-jdbc:12.8.1.jre11")
     implementation("org.mariadb.jdbc:mariadb-java-client:3.5.2")
+    implementation("org.postgresql:postgresql:42.7.4")
+
+    testImplementation(platform("org.junit:junit-bom:5.11.4"))
+    testImplementation("org.junit.jupiter:junit-jupiter")
+    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
+
+    // Engine integration tests run against real databases (cross-vendor: PostgreSQL -> MariaDB)
+    // via Testcontainers - H2/mocks would miss dialect, transaction and driver behaviour. The
+    // JDBC drivers themselves are already on the classpath (implementation, above).
+    testImplementation(platform("org.testcontainers:testcontainers-bom:1.20.4"))
+    testImplementation("org.testcontainers:junit-jupiter")
+    testImplementation("org.testcontainers:postgresql")
+    testImplementation("org.testcontainers:mariadb")
+}
+
+// Compile to Java 21 bytecode (virtual threads are stable since 21) so the app runs on any
+// JDK 21+ without juggling JAVA_HOME. Running it on JDK 24+/25 additionally avoids
+// virtual-thread pinning in JDBC drivers (JEP 491) - recommended for best throughput, but
+// not required.
+java {
+    toolchain {
+        languageVersion = JavaLanguageVersion.of(21)
+    }
+}
+
+tasks.withType<JavaCompile>().configureEach {
+    options.encoding = "UTF-8"
+}
+
+// The default i18n/ and branding/ resources live in src/main/resources and are bundled into the
+// jar automatically. application.example.yml stays at the project root as a user-facing config
+// template (.env.example style), so copy that one into the jar for the first-run seed + init.
+// External files (next to the jar, or in the CWD) still override these - see Home / the loaders.
+tasks.processResources {
+    from("application.example.yml")
+}
+
+tasks.test {
+    useJUnitPlatform()
 }
 
 application {
     mainClass = "kr.devslab.datalinq.Main"
+    // Korean content + Windows console: force UTF-8 stdout/stderr.
+    // --enable-native-access: JLine uses native terminal access (silences the Java 25
+    // restricted-method warning and stays working when it becomes enforced).
+    applicationDefaultJvmArgs = listOf(
+        "-Dstdout.encoding=UTF-8",
+        "-Dstderr.encoding=UTF-8",
+        "--enable-native-access=ALL-UNNAMED",
+    )
 }
 
-// The TUI reads from the real terminal; let `gradle run` pass stdin through.
+// The TUI reads from the real terminal; let `gradle run` / `gradle runShadow` pass stdin through.
 tasks.named<JavaExec>("run") {
     standardInput = System.`in`
 }
+tasks.named<JavaExec>("runShadow") {
+    standardInput = System.`in`
+}
+
+// The droppable deliverable: a single self-contained jar. mergeServiceFiles() is essential -
+// the JDBC drivers (java.sql.Driver) and our MigrationHandler register via ServiceLoader, and a
+// naive fat jar would keep only one service file. Stripping signature files avoids the signed-jar
+// SecurityException when driver jars are merged.
+tasks.shadowJar {
+    archiveBaseName = "datalinq"
+    archiveClassifier = ""
+    archiveVersion = ""
+    // duplicatesStrategy takes precedence over transformers: the default would drop the 2nd
+    // META-INF/services/java.sql.Driver (MariaDB's) before mergeServiceFiles() sees it, leaving
+    // only the MSSQL driver registered. INCLUDE lets the transformer merge both.
+    duplicatesStrategy = DuplicatesStrategy.INCLUDE
+    mergeServiceFiles()
+    exclude("META-INF/*.SF", "META-INF/*.DSA", "META-INF/*.RSA")
+}
+
+// Bundle the editable default resources into the distribution image (alongside bin/ and lib/)
+// so the launcher resolves them via Home (the install dir) and works when run from anywhere.
+// application.yml is intentionally NOT bundled - it is gitignored and holds credentials; the
+// app seeds from application.example.yml on first run and saves edits to application.yml.
+distributions {
+    named("main") {
+        contents {
+            from("i18n") { into("i18n") }
+            from("branding") { into("branding") }
+            from("sql") { into("sql") }
+            from("application.example.yml")
+        }
+    }
+}