From 164161664ab14816b5b8543e6c179679301c86b0 Mon Sep 17 00:00:00 2001 From: Bibo Hao Date: Thu, 25 Jun 2026 16:24:21 +0800 Subject: [PATCH 1/4] add skills for agents --- .agents/skills | 1 + .claude/skills | 1 + .github/copilot-instructions.md | 1 + .github/skills | 1 + AGENTS.md | 0 CLAUDE.md | 1 + doc/skills/project_scaffolding.md | 32 +++++++++++++++++++++++++++++++ 7 files changed, 37 insertions(+) create mode 120000 .agents/skills create mode 120000 .claude/skills create mode 120000 .github/copilot-instructions.md create mode 120000 .github/skills create mode 100644 AGENTS.md create mode 120000 CLAUDE.md diff --git a/.agents/skills b/.agents/skills new file mode 120000 index 0000000..cd2ebc5 --- /dev/null +++ b/.agents/skills @@ -0,0 +1 @@ +../doc/skills \ No newline at end of file diff --git a/.claude/skills b/.claude/skills new file mode 120000 index 0000000..cd2ebc5 --- /dev/null +++ b/.claude/skills @@ -0,0 +1 @@ +../doc/skills \ No newline at end of file diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 120000 index 0000000..be77ac8 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1 @@ +../AGENTS.md \ No newline at end of file diff --git a/.github/skills b/.github/skills new file mode 120000 index 0000000..cd2ebc5 --- /dev/null +++ b/.github/skills @@ -0,0 +1 @@ +../doc/skills \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..e69de29 diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000..47dc3e3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/doc/skills/project_scaffolding.md b/doc/skills/project_scaffolding.md index 0fd7634..19a241e 100644 --- a/doc/skills/project_scaffolding.md +++ b/doc/skills/project_scaffolding.md @@ -16,6 +16,11 @@ The `aloha-python` repository is organized into several top-level directories, e - **`tool/`**: This directory contains development and CI/CD related scripts and Docker assets. This includes scripts for setting up the development environment, building Docker images, and managing the project lifecycle. +- **`AGENTS.md`**: Centralized file containing project-wide guidelines, rules, and style constraints shared across various AI agents. +- **`.agents/`**: Folder for Antigravity-specific customizations. The `skills/` subdirectory is symlinked to `doc/skills/` to load skills. +- **`.claude/`**: Folder for Claude Code-specific configurations. The `skills/` subdirectory is symlinked to `doc/skills/` to provide skills/instructions. +- **`.github/`**: Contains GitHub Actions workflows (`.github/workflows/`), Copilot instructions (`.github/copilot-instructions.md`, symlinked to `AGENTS.md`), and a symlink `.github/skills` to `../doc/skills`. + ## 2. Using the Repository as a Boilerplate To initiate a new project using `aloha-python` as a boilerplate, follow these steps: @@ -47,3 +52,30 @@ To initiate a new project using `aloha-python` as a boilerplate, follow these st ``` By adhering to these conventions, AI agents can effectively understand, navigate, and contribute to projects built upon the `aloha-python` framework. + +## 4. Multi-Agent Setup and Scaffolding Script + +To facilitate cooperation and consistent behavior across multiple AI tools and agents (such as Antigravity/Gemini, Claude Code, and GitHub Copilot), the project uses a unified instruction layout. The following scaffolding script initializes this structure: + +```bash +# Create scaffolding directories +mkdir -pv .github/workflows doc/skills src/tests tool/cicd + +# Create agent config directories and symlink the centralized skills directory +mkdir -pv doc/skills .agents .claude \ +&& touch AGENTS.md \ +&& ln -sf AGENTS.md CLAUDE.md \ +&& ln -sf ../AGENTS.md .github/copilot-instructions.md \ +&& ln -sf ../doc/skills .agents/ \ +&& ln -sf ../doc/skills .claude/ \ +&& ln -sf ../doc/skills .github/ +``` + +### Key Components of the Multi-Agent Framework: +- **Centralized Agent Rules (`AGENTS.md`)**: + - `AGENTS.md` is the single source of truth for guidelines, code style constraints, and project rules. + - `CLAUDE.md` is symlinked to `AGENTS.md` for Claude Code. + - `.github/copilot-instructions.md` is symlinked to `../AGENTS.md` for GitHub Copilot. +- **Centralized Agent Skills (`doc/skills/`)**: + - Agent skills and instruction documents (like this one) are stored in the `doc/skills/` directory. + - Symlinks are created under `.agents/skills`, `.claude/skills`, and `.github/skills` pointing to `../doc/skills`. This ensures that all agents read the same skill instructions. From 5c870c88325893a76ba9e2c8a1e4313a0ecc1be4 Mon Sep 17 00:00:00 2001 From: Bibo Hao Date: Thu, 25 Jun 2026 20:37:51 +0800 Subject: [PATCH 2/4] update skills --- doc/skills/aloha_cicd/SKILL.md | 63 +++++++++++ doc/skills/aloha_cicd/references/dev_env.md | 40 +++++++ .../aloha_cicd/references/scaffolding.md | 41 +++++++ doc/skills/aloha_package_usage.md | 102 ------------------ doc/skills/aloha_python/SKILL.md | 76 +++++++++++++ doc/skills/aloha_python/references/compile.md | 55 ++++++++++ doc/skills/aloha_python/references/config.md | 65 +++++++++++ doc/skills/aloha_python/references/db.md | 45 ++++++++ doc/skills/aloha_python/references/encrypt.md | 67 ++++++++++++ doc/skills/aloha_python/references/logger.md | 55 ++++++++++ doc/skills/aloha_python/references/testing.md | 24 +++++ doc/skills/dev_env_cicd.md | 72 ------------- doc/skills/project_scaffolding.md | 81 -------------- 13 files changed, 531 insertions(+), 255 deletions(-) create mode 100644 doc/skills/aloha_cicd/SKILL.md create mode 100644 doc/skills/aloha_cicd/references/dev_env.md create mode 100644 doc/skills/aloha_cicd/references/scaffolding.md delete mode 100644 doc/skills/aloha_package_usage.md create mode 100644 doc/skills/aloha_python/SKILL.md create mode 100644 doc/skills/aloha_python/references/compile.md create mode 100644 doc/skills/aloha_python/references/config.md create mode 100644 doc/skills/aloha_python/references/db.md create mode 100644 doc/skills/aloha_python/references/encrypt.md create mode 100644 doc/skills/aloha_python/references/logger.md create mode 100644 doc/skills/aloha_python/references/testing.md delete mode 100644 doc/skills/dev_env_cicd.md delete mode 100644 doc/skills/project_scaffolding.md diff --git a/doc/skills/aloha_cicd/SKILL.md b/doc/skills/aloha_cicd/SKILL.md new file mode 100644 index 0000000..8b890a1 --- /dev/null +++ b/doc/skills/aloha_cicd/SKILL.md @@ -0,0 +1,63 @@ +--- +name: aloha-cicd +description: Guides on project scaffolding, multi-agent setup, local containerized development via run-dev.sh, and building Docker images for production. +--- + +# Aloha Project Scaffolding and Docker CI/CD Skill + +This skill outlines the process for setting up boilerplate codebases, orchestrating containerized local development environments, and building production Docker images. + +--- + +## 1. Project Scaffolding & Multi-Agent setup + +To set up your project folders and ensure consistent AI assistant behaviors (across Antigravity, Claude Code, and Copilot), refer to the **[Project Scaffolding & Multi-Agent Setup reference](file:///home/haobibo/dev/dev-labnow/aloha-python/doc/skills/aloha_cicd/references/scaffolding.md)**. This guide covers: +- Core directory layout conventions (`src/`, `doc/`, `notebook/`, `tool/`). +- Handing library isolation (excluding `pkg/` from new app template repositories). +- Centralized rules (`AGENTS.md`) and centralized skills (`doc/skills/`) configuration via symlinks. + +--- + +## 2. Local Development Environment + +To start, run, and enter the development container with auto-allocated user ports and volume mappings, refer to the **[Local Development Container & Compose reference](file:///home/haobibo/dev/dev-labnow/aloha-python/doc/skills/aloha_cicd/references/dev_env.md)**. This guide covers: +- Lifecycle management using `./tool/cicd/run-dev.sh` (`up`, `enter`, `logs`, `restart`, `down`). +- User-specific dynamic port calculation (`PORT_APP` and `PORT_WEB`). +- Mounting credentials and setting the `PYTHONPATH`. +- Specs for `dev-demo.Dockerfile` and shared DB compose configurations. + +--- + +## 3. Production Compilation & Packaging + +To compile your code into binary extension libraries (`.so` files) and package them into a lightweight container image: + +1. **Production Dockerfile ([src/app-demo.Dockerfile](file:///home/haobibo/dev/dev-labnow/aloha-python/src/app-demo.Dockerfile))**: + - Uses a two-stage build layout. + - Converts normal Python `.py` files into binary dynamic libraries inside the builder stage (using `aloha compile`) if `ENABLE_CODE_BUILD="true"`. + - Packs compiled assets into a clean minimal environment, exposing `PORT_SVC` (default 80) and launching via `aloha start`. +2. **Build Trigger Command**: + ```bash + source tool/tool.sh + build_image app_common latest src/app-demo.Dockerfile + ``` + +--- + +## 4. Running Tests & Code Coverage + +Tests are executed using `pytest` inside the running dev container: + +1. Start and enter the development container: + ```bash + ./tool/cicd/run-dev.sh up + ./tool/cicd/run-dev.sh enter + ``` +2. Run your tests from the root directory: + ```bash + # Run all unit and integration tests + pytest src/ + + # Run tests with code coverage report + pytest --cov=src src/ + ``` diff --git a/doc/skills/aloha_cicd/references/dev_env.md b/doc/skills/aloha_cicd/references/dev_env.md new file mode 100644 index 0000000..48b726f --- /dev/null +++ b/doc/skills/aloha_cicd/references/dev_env.md @@ -0,0 +1,40 @@ +# Local Development Container & Docker Compose + +This sub-module guides you through setting up and using the containerized local development environment via `docker-compose` and the `run-dev.sh` lifecycle helper. + +--- + +## 1. Local Development Containerization (`run-dev.sh`) + +To guarantee environment consistency, local development is fully containerized. The `./tool/cicd/run-dev.sh` script manages this container. + +### Commands + +| Command | Action | +| :--- | :--- | +| `./tool/cicd/run-dev.sh up` | Checks port availability and starts the development container in detached mode. | +| `./tool/cicd/run-dev.sh enter` | Spawns an interactive bash terminal inside the running container. | +| `./tool/cicd/run-dev.sh logs` | Tails and streams container logs to stdout. | +| `./tool/cicd/run-dev.sh restart` | Restarts the development container. | +| `./tool/cicd/run-dev.sh down` | Stops and removes the development container. | + +### Configuration Details + +- **User-Specific Port Mapping**: To prevent port clashes on shared, multi-user systems, host ports are computed dynamically using the user's numeric ID (`UID`): + - **Application Port (`PORT_APP`)**: `30000 + UID` (exposes port 9000). + - **Web Port (`PORT_WEB`)**: `33000 + UID` (exposes port 3000). +- **Import Environments**: The container mounts the project root to `/root/app:rw` and automatically sets `PYTHONPATH=/root/app/pkg:/root/app/src:/root/app/notebook` for clean imports. +- **Agent Credentials Mounts**: Volume-mounts local config directories (`~/.gemini`, `~/.claude`, `~/.copilot`) to keep credential cache and auth profiles active. + +--- + +## 2. Docker Compose & Dockerfiles Specs + +- **Development Dockerfile ([src/dev-demo.Dockerfile](file:///home/haobibo/dev/dev-labnow/aloha-python/src/dev-demo.Dockerfile))**: + - Defines the base environment for development. + - Installs Node.js (`pnpm`), Python (`jupyterlab`), package requirements from `src/requirements.txt`, and PostgreSQL client CLI. +- **Development Compose ([tool/cicd/docker-compose.app-demo.DEV.yml](file:///home/haobibo/dev/dev-labnow/aloha-python/tool/cicd/docker-compose.app-demo.DEV.yml))**: + - Configures the development service container, exposing computed ports, mounting workspace files, and setting environment variables. +- **Common Database Server ([tool/cicd/docker-compose.db.yml](file:///home/haobibo/dev/dev-labnow/aloha-python/tool/cicd/docker-compose.db.yml))**: + - Launches a shared PostgreSQL 17 database service (`db-postgres-common`) on port 5432. + - Preloaded with database extensions: `pg_duckdb`, `pg_search`, `pg_cron`. diff --git a/doc/skills/aloha_cicd/references/scaffolding.md b/doc/skills/aloha_cicd/references/scaffolding.md new file mode 100644 index 0000000..32734a1 --- /dev/null +++ b/doc/skills/aloha_cicd/references/scaffolding.md @@ -0,0 +1,41 @@ +# Project Scaffolding & Multi-Agent Setup + +This sub-module guides you through setting up project directories, directory conventions, and configuring a multi-agent workspace. + +--- + +## 1. Directory Structure & Conventions + +When using `aloha-python` as a boilerplate for a new project, follow these structural rules: + +- **`src/` (Core Application & Tests)**: Place all application-specific code and tests here. + - **`src/resource/config/`**: HOCON configuration files (e.g. `main.conf`, `deploy-DEV.conf`). + - **`src/tests/`**: All test files (prefixed with `test_*.py`) and test resources. +- **`doc/` (Documentation & Agent Skills)**: Project Markdown documentation and AI Agent Skills (stored under `doc/skills/`). +- **`notebook/` (Jupyter Notebooks)**: For data analysis, rapid prototyping, and interactive experimentation. +- **`tool/` (CI/CD & Shell Helpers)**: Holds docker compose setups, dockerfiles, and helper scripts. +- **`pkg/` (Library Source - Exclude in Boilerplate Apps)**: This is the core `aloha` package itself. Do not clone/copy the `pkg/` directory when creating a new application project based on this template; your application code should live in `src/`. + +--- + +## 2. Multi-Agent Scaffolding Script + +To ensure consistent guidelines, style rules, and skill access across different AI tools (such as Antigravity/Gemini, Claude Code, and GitHub Copilot), run the following scaffolding commands to link config directories: + +```bash +# Create scaffolding directories +mkdir -pv .github/workflows doc/skills src/tests tool/cicd + +# Create agent config directories and symlink the centralized rules and skills +mkdir -pv doc/skills .agents .claude \ + && touch AGENTS.md \ + && ln -sf AGENTS.md CLAUDE.md \ + && ln -sf ../AGENTS.md .github/copilot-instructions.md \ + && ln -sf ../doc/skills .agents/ \ + && ln -sf ../doc/skills .claude/ \ + && ln -sf ../doc/skills .github/ +``` + +### Integrated Workspace Layout +- **Centralized Rules (`AGENTS.md`)**: The single source of truth for code styling, conventions, and constraints. +- **Centralized Skills (`doc/skills/`)**: Symlinked dynamically to `.agents/skills`, `.claude/skills`, and `.github/skills`. All agents automatically share the same custom instructions. diff --git a/doc/skills/aloha_package_usage.md b/doc/skills/aloha_package_usage.md deleted file mode 100644 index c6d5709..0000000 --- a/doc/skills/aloha_package_usage.md +++ /dev/null @@ -1,102 +0,0 @@ -# Aloha Package Usage and API Development Skill - -This skill provides guidance on how to effectively use the `aloha` Python package for developing applications within the `aloha-python` boilerplate structure. - -## 1. Importing and Using the `aloha` Package - -The `aloha` package is designed to be easily integrated into your Python projects. Once installed (e.g., `pip install aloha[all]`), you can import its modules as needed. - -### Example: Basic `aloha` Module Usage - -To use `aloha` modules, you simply import them and use their functionalities. For example, using `aloha.logger`: - -```python -from aloha.logger import LOG - -def my_function(): - LOG.info("This is an informational message from aloha logger.") - return "Function executed successfully" -``` - -In this example: -- `aloha.logger.LOG` provides a pre-configured logger instance. - -## 2. Application Configuration with HOCON - -`aloha` applications are configured using HOCON (Human-Optimized Config Object Notation) files, typically located in `src/resource/config/`. The `aloha.config.paths` module helps in discovering these configuration files. - -### Configuration File Location and Structure - -By default, `aloha` looks for configuration files in the `src/resource/config/` directory. The primary configuration file is `main.conf`. - -HOCON allows for hierarchical, modular, and human-readable configurations. You can include other configuration files, which is useful for modularizing your settings (e.g., separating development, staging, and production configurations). - -**Example `main.conf`:** - -```hocon -include "deploy-DEV.conf" - -app_name = "MyAlohaApp" - -server { - host = "0.0.0.0" - port = 8080 -} - -database { - type = "postgresql" - connection_string = "${?DB_CONNECTION_STRING}" # Environment variable override -} -``` - -In this example: -- `include "deploy-DEV.conf"` brings in settings from another file, allowing for environment-specific overrides. -- `app_name` and `server` define application-specific settings. -- `database.connection_string = "${?DB_CONNECTION_STRING}"` demonstrates how to use environment variables to override configuration values, making it flexible for different deployment environments. - -### Accessing Configuration in Code - -You can access configuration values in your Python code via `aloha.settings.SETTINGS.config`: - -```python -from aloha.settings import SETTINGS - -config = SETTINGS.config -print(f"Application Name: {config.get("app_name")}") -print(f"Server Port: {config.get("server.port")}") -``` - -This approach ensures that your application remains configurable and adaptable to various deployment scenarios. - -## 3. Running Your `aloha`-based Application - -To run your `aloha`-based application, you can use the `src/main.py` script, which acts as a generic module runner: - -### Example: Application Entry Point - -Your application's main entry point should be a function (e.g., `main()`) within a module in your `src/` directory. For example, in `src/my_app/main.py`: - -```python -def main(): - # Your application's startup logic here - print("My Aloha application is starting...") - # Integrate with FastAPI or other frameworks here -``` - -### Running the Application - -To execute your application, use the `src/main.py` script: - -```bash -python3 src/main.py my_app.main -``` - -This command tells `src/main.py` to find and execute the `main()` function within your specified module (e.g., `my_app.main`). - -## 4. Advanced Usage and Best Practices - -- **Configuration Management**: Leverage `aloha.config.paths` and HOCON files for environment-specific configurations (e.g., `deploy-DEV.conf`). This allows for modular and flexible management of settings across different environments. -- **HOCON Configuration Details**: For a comprehensive understanding of HOCON configuration, refer to the "Configuration with HOCON" section in the `README-get-start.md` documentation. -- **Database Integration**: Utilize `aloha.db` modules (e.g., `aloha.db.postgres`) for seamless database interactions, as demonstrated in `src/app_common/api/api_common_query_postgres.py`. -- **Logging**: Use `aloha.logger.LOG` for consistent and configurable logging across your application. -- **Testing**: Employ `aloha.testing` utilities for unit and integration tests. diff --git a/doc/skills/aloha_python/SKILL.md b/doc/skills/aloha_python/SKILL.md new file mode 100644 index 0000000..bf5f95c --- /dev/null +++ b/doc/skills/aloha_python/SKILL.md @@ -0,0 +1,76 @@ +--- +name: aloha-python +description: Guides on Python development standards, importing and using the aloha package, HOCON configuration, and running aloha-based applications. +--- + +# Aloha Python Skill + +This skill provides coding standards, modular application structures, and usage guidelines for the `aloha` Python package within the `aloha-python` boilerplate structure. + +--- + +## 1. Python Naming & Coding Standards + +When developing Python code in this codebase, adhere to the following naming conventions: + +- **Primary Type/First-Class Identity Prefix**: Place the variable's type, role, or primary characteristics first in its name. + - _Correct_: `name_service`, `port_service`, `svc_ingress`, `cfg_postgres`. + - _Incorrect_: `service_name`, `service_port`, `ingress_service`, `postgres_config`. +- **Logger naming**: Use lowercase with underscores for logger names, e.g., `db_sync`. + +--- + +## 2. Package Architecture & Sub-Modules + +The `aloha` package is divided into several specialized sub-modules. See their respective reference documents for detailed APIs and usage examples: + +- **[Configuration Management (`aloha.config`)](references/config.md)**: HOCON configuration loading, environment profile handling (`ENV_PROFILE`), and settings loading via `SETTINGS`. +- **[Logging Framework (`aloha.logger`)](references/logger.md)**: Safe concurrent multi-process logging, console handlers, and custom logger setups. +- **[Encryption & Hashing (`aloha.encrypt`)](references/encrypt.md)**: AES encryption, RSA keys generation & signing/verifying, JWT encoding/decoding, and dictionary/object hashing helpers. +- **[Testing Utilities (`aloha.testing`)](references/testing.md)**: Base `UnitTestCase` class with pre-configured settings/loggers, and `ServiceTestCase` for testing HTTP API endpoints. +- **[Database Operators (`aloha.db`)](references/db.md)**: Secure DB operations through SQLAlchemy-backed operators (e.g. `PostgresOperator`), with password resolution via `PasswordVault`. +- **[Binary Compilation (`aloha compile`)](references/compile.md)**: Compiling normal `.py` files into binary dynamic library files using Cython for intellectual property protection. + +--- + +## 3. Running Your `aloha`-based Application + +Your application's main entry point should be a function (e.g., `main()`) within a module inside the `src/` directory. + +### Running with `src/main.py` + +`src/main.py` acts as a generic module runner. Run modules using: + +```bash +python3 src/main.py . +``` + +**Example**: + +```bash +python3 src/main.py my_app.main +``` + +This command resolves and executes the `main()` function in `src/my_app/main.py`. + +--- + +## 4. Compiling Code into Binary + +For IP protection, you can build your package into compiled dynamic library files (such as `.so` files on Linux/macOS) using Cython. + +### Compilation Command + +```bash +aloha compile --base= --dist= --keep='' +``` + +For full instructions, parameter reference, and package structural requirements (e.g. `__init__.py` usage), see the **[Binary Compilation reference](references/compile.md)**. + +--- + +## 5. Summary of Advanced Best Practices + +- **Configuration Isolation**: Keep config files modular by splitting environment configurations (e.g., `deploy-DEV.conf`, `deploy-PROD.conf`) and loading them using HOCON `include` syntax in `main.conf`. +- **Concurrency Logging Safety**: Avoid standard `FileHandler` in parallel/multi-process tasks; always use `aloha.logger.LOG` or named loggers from `get_logger()`. +- **Database Credentials**: Never hardcode passwords in config files. Configure them to resolve dynamically through the `PasswordVault`. diff --git a/doc/skills/aloha_python/references/compile.md b/doc/skills/aloha_python/references/compile.md new file mode 100644 index 0000000..c4d5380 --- /dev/null +++ b/doc/skills/aloha_python/references/compile.md @@ -0,0 +1,55 @@ +# Python Binary Compilation Module (`aloha compile`) + +The `aloha compile` (or `python -m aloha.script.compile`) utility compiles Python source code (`.py`) into platform-specific compiled extension libraries (such as `.so` on Linux/macOS or `.pyd` on Windows) using Cython. This process helps protect intellectual property by hiding implementation details and preventing direct inspection of source code. + +## 1. CLI Command Usage + +To compile your application, run: + +```bash +aloha compile --base= --dist= --keep='' +``` + +### Example +```bash +aloha compile --base=./src --dist=./dist_build --keep='main.py' +``` + +--- + +## 2. Compilation Workflow & Logic + +Under the hood, `aloha compile` executes the following steps: + +1. **Source Discovery**: Scans the source directory specified by `--base` (defaults to the current working directory). +2. **Filter & Skip**: Skips hidden files/directories (starting with `.`), the target `--dist` folder, and paths specified by `--exclude`. +3. **Asset Copying**: Copies all non-Python files (e.g. configurations, static assets) as-is to the target structure. +4. **Cythonization**: + - Converts normal `.py` files into C files. + - Compiles these C files into platform-native dynamic library files (using parallel compilation on multiple CPU threads). + - Skips `__init__.py` files (they are always copied as plain Python files to maintain package structure). + - Skips files specified in `--keep` (they remain as plain `.py` files). +5. **Temporary Storage & Final Cleanup**: Writes temporary intermediate files to `/tmp/build/`, then moves the final compiled outputs to `--dist` and deletes the temporary files. + +--- + +## 3. Important Caveats & Best Practices + +- **Cython Prerequisite**: Cython must be installed in your environment prior to running the command: + ```bash + pip install Cython + ``` +- **`__init__.py` Requirement**: All directories acting as Python submodules/packages **must** contain an `__init__.py` file. If a directory lacks `__init__.py`, the compiled extension files under it will be incorrectly placed in the root of the output directory. +- **Empty Output Directory**: The folder path specified by `--dist` must either not exist or be completely empty. The compilation script will raise an error if the directory contains existing files to prevent accidental overwrites. +- **Excluded files**: You can pass one or more file patterns to `--exclude` to bypass compile/copy of test suites, temporary files, or local dev tools. + +--- + +## 4. CLI Parameter Reference + +| Parameter | Type | Default | Description | +| :--- | :--- | :--- | :--- | +| `--base` | string | `./` | Root folder containing the Python source code to compile. | +| `--dist` | string | `build` | Destination directory where the final binary files and copied assets will be placed. | +| `--exclude` | string list | `()` | A list of file or folder path glob patterns to exclude from being copied or compiled. | +| `--keep` | string list | `()` | A list of specific `.py` files to keep as plain Python source code instead of compiling. | diff --git a/doc/skills/aloha_python/references/config.md b/doc/skills/aloha_python/references/config.md new file mode 100644 index 0000000..d10e688 --- /dev/null +++ b/doc/skills/aloha_python/references/config.md @@ -0,0 +1,65 @@ +# Configuration Management Module (`aloha.config`) + +The `aloha.config` subpackage provides tools for discovering, loading, and parsing application configurations written in HOCON (Human-Optimized Config Object Notation). The configuration process is highly driven by Operating System Environment Variables. + +--- + +## 1. Operating System Environment Variables + +The configuration and startup behavior of an `aloha` application can be customized using the following environment variables: + +| Environment Variable | Default Value | Description | +| :--- | :--- | :--- | +| `ENV_PROFILE` | `None` (undefined) | Specifies the running profile (e.g., `DEV`, `STG`, `PRD`). Determines the configuration entry file (`main-${ENV_PROFILE}.conf`). | +| `ENTRYPOINT` | `None` (undefined) | Specifies the default Python module entry point when running `aloha start`. The module must contain a `main()` function. | +| `APP_MODULE` | `default` | Identifies the application/module name. Mapped to configuration key `APP_MODULE` and used as a prefix for the log file names. | +| `DIR_LOG` | `logs` | The directory where log files are stored. | +| `DIR_RESOURCE` | `resource` (under CWD) | Root directory containing non-code resources (e.g. assets, static data). | +| `DIR_CONFIG` | `${DIR_RESOURCE}/config` | Directory where configuration files are located. | +| `FILES_CONFIG` | `None` (undefined) | Comma-separated list of configuration filenames to load (e.g., `db.conf,server.conf`). If defined, it overrides `ENV_PROFILE`. | + +--- + +## 2. Path Discovery (`aloha.config.paths`) + +This module resolves paths for config directories, resource directories, and active configuration files. + +### Key Functions +- `get_resource_dir(*args) -> str`: Resolves the absolute path to the resource directory. Relies on the `DIR_RESOURCE` environment variable. +- `get_config_dir(*args) -> str`: Resolves the absolute path to the configuration directory. Relies on the `DIR_CONFIG` environment variable. +- `get_config_files() -> list`: Determines which HOCON configuration files should be loaded. + - If `FILES_CONFIG` environment variable is defined, it splits the list by comma and resolves their paths. + - If `FILES_CONFIG` is not defined but `ENV_PROFILE` is defined, it resolves `main-${ENV_PROFILE}.conf`. + - Otherwise, it defaults to `main.conf`. +- `get_project_base_dir(file_caller: str) -> str`: Traverses directories upwards from `file_caller` (typically passed as `__file__`) until it finds a directory containing no `__init__.py`, marking the project base root. + +--- + +## 3. HOCON Loading (`aloha.config.hocon`) + +This module uses the `pyhocon` library to parse configurations. + +### Key Functions +- `load_config_from_hocon(config_file: str) -> dict`: Parses a single HOCON file into a plain ordered dictionary. +- `load_config_from_hocon_files(config_files: list, base_dir: str) -> AttrDict`: Generates a string containing HOCON `include required("")` directives for each configuration file, parses it within the context of `base_dir`, and returns the merged configuration as an `AttrDict` object. This enables attribute-style dot access. + +--- + +## 4. Settings Interface (`aloha.settings`) + +The `aloha.settings` module exports a global, pre-instantiated singleton `SETTINGS` of class `Settings`, which manages lazy loading of config files. + +### Usage Example +```python +from aloha.settings import SETTINGS + +# Access the global configuration (loads configuration lazily on first access) +config = SETTINGS.config + +# Dot-style attribute access (since config is an AttrDict) +app_name = config.app_name +db_host = config.database.host + +# Or retrieve specific paths as dictionary keys +port = config.get("server.port", 8080) +``` diff --git a/doc/skills/aloha_python/references/db.md b/doc/skills/aloha_python/references/db.md new file mode 100644 index 0000000..4f9774f --- /dev/null +++ b/doc/skills/aloha_python/references/db.md @@ -0,0 +1,45 @@ +# Database Operators Module (`aloha.db`) + +The `aloha.db` subpackage provides operators to connect and perform queries across databases, including PostgreSQL, MySQL, SQLite, Redis, DuckDB, MongoDB, Elasticsearch, and Kafka. + +## 1. Password Vault Integration (`aloha.db.base`) + +Connection configurations securely resolve passwords through `PasswordVault` (supporting standard text, vault servers, or custom integrations). + +```python +from aloha.db.base import PasswordVault + +# Resolves vault config to extract a password string +vault = PasswordVault.get_vault(cfg_db.get("vault_type"), cfg_db.get("vault_config")) +password = vault.get_password(cfg_db.get("password")) +``` + +--- + +## 2. PostgreSQL Operator (`aloha.db.postgres`) + +Creates and manages an SQLalchemy connection pool backed by `psycopg`. + +### Key Classes + +- `PostgresOperator(cfg_db: dict, **kwargs)` + - Methods: + - `execute_query(sql: str, *args, **kwargs) -> CursorResult`: Executes a raw SQL command inside a connection context block. + - Properties: + - `connection`: Returns the SQLAlchemy `Engine` instance. + - `connection_str -> str`: Returns a connection string with credentials hidden or resolved. + +### Usage Example + +```python +from aloha.db.postgres import PostgresOperator +from aloha.settings import SETTINGS + +cfg_db = SETTINGS.config.get("db_postgres") +postgres_op = PostgresOperator(cfg_db) + +# Run raw SQL +result = postgres_op.execute_query("SELECT NOW();") +for row in result: + print(row) +``` diff --git a/doc/skills/aloha_python/references/encrypt.md b/doc/skills/aloha_python/references/encrypt.md new file mode 100644 index 0000000..ca838a3 --- /dev/null +++ b/doc/skills/aloha_python/references/encrypt.md @@ -0,0 +1,67 @@ +# Encryption and Utility Modules (`aloha.encrypt`) + +The `aloha.encrypt` subpackage contains functions and classes for AES encryption, RSA signing/encryption, JWT token encoding/decoding, and hashing utilities. + +## 1. AES Encryption (`aloha.encrypt.aes`) + +Uses PyCryptodome to encrypt/decrypt payloads with standard AES configurations. + +### Key Classes + +- `AesEncryptor(key: Union[str, bytes] = None, key_size: int = 16, cipher_name: str = "AES/ECB/PKCS5Padding")` + - Supports ECB and CBC modes (defined in `supported_cipher_methods`). + - Methods: + - `encrypt(text: str, output_format="hex", func_pad=None) -> Union[str, bytes]` + - `decrypt(text: Union[str, bytes], input_format="hex", func_unpad=None) -> str` + +### Usage Example + +```python +from aloha.encrypt.aes import AesEncryptor + +encryptor = AesEncryptor(key="my-secret-16bytes", cipher_name="AES/CBC/PKCS7Padding") +ciphertext = encryptor.encrypt("Hello World") +decrypted = encryptor.decrypt(ciphertext) +``` + +--- + +## 2. RSA Cryptography (`aloha.encrypt.rsa`) + +Provides asymmetric encryption, decryption, and signature creation/verification using RSA key pairs. + +### Key Classes + +- `RsaEncryptor(key_private: str | None = None, key_public: str | None = None, cipher_name: str = "RSA/ECB/PKCS1Padding")` + - Methods: + - `encrypt(text: str, output_format="base64") -> str` + - `decrypt(text: str, input_format="base64") -> str` + - `sign(text: str, signature_format="base64", hash_algo="SHA-256") -> str` + - `verify(text: str, signature: str, signature_format="base64", hash_algo="SHA-256") -> bool` + - Static Methods: + - `generate_key_pair(size: int = 1024) -> Tuple[str, str]`: Generates a tuple of `(private_key_pem, public_key_pem)`. + +--- + +## 3. JWT Utilities (`aloha.encrypt.jwt`) + +Wraps `pyjwt` to encode and decode tokens. + +### Key Functions + +- `encode(secret_key: str, payload: dict, headers: dict | None = None, **kwargs) -> str` +- `decode(secret_key: str, token: str, **kwargs) -> Union[dict, str]`: Decodes the token, catching exceptions like `ExpiredSignatureError` or `PyJWTError` and returning them as strings. + +--- + +## 4. Hashing Utilities (`aloha.encrypt.hash`) + +Provides standard hashing algorithms. + +### Key Functions + +- `get_md5_of_str(string: str) -> str`: Return MD5 hex digest. +- `get_sha256_of_str(string: str) -> str`: Return SHA-256 hex digest. +- `hash_dict(dic: dict) -> str`: Normalizes the dictionary to sorted JSON and returns its MD5 hash. +- `hash_obj(obj: any) -> str`: Serializes any JSON-compatible object and returns its MD5 hash. +- `hash_base62(s: str, length: int = 6) -> str`: Computes SHA-256 and encodes it using a custom Base62 alphabet of length between 1 and 11. diff --git a/doc/skills/aloha_python/references/logger.md b/doc/skills/aloha_python/references/logger.md new file mode 100644 index 0000000..fff368c --- /dev/null +++ b/doc/skills/aloha_python/references/logger.md @@ -0,0 +1,55 @@ +# Logging Module (`aloha.logger`) + +The `aloha.logger` subpackage provides a pre-configured, multi-process safe logging framework. + +## 1. Global Logger (`LOG`) + +The module exports a global pre-configured logger named `LOG`. It reads log levels from settings under `deploy.log_level` (falling back to `logging.DEBUG` if unset). + +### Usage Example + +```python +from aloha.logger import LOG + +def run_task(): + LOG.debug("Starting task processing...") + try: + # Task implementation + LOG.info("Task completed successfully.") + except Exception as e: + LOG.error(f"Task failed: {e}", exc_info=True) +``` + +--- + +## 2. Dynamic Logger Creation (`get_logger`) + +You can create named loggers dynamically with customized levels. + +### Key Functions + +- `get_logger(logger_name: str | None = None, level=logging.DEBUG, **kwargs) -> logging.Logger`: Retrieves and configures a logger. +- `getLogger`: An alias to `get_logger`. + +### Usage Example + +```python +from aloha.logger import get_logger + +logger_custom = get_logger("database_sync", level="INFO") +logger_custom.info("Custom sync logger initialized.") +``` + +--- + +## 3. Implementation Details + +- **Safe Concurrent File Writes**: Utilizes `MultiProcessSafeDailyRotatingFileHandler` to avoid lock conflicts or log corruption when multiple parallel processes write logs concurrently. +- **Log Location**: Writes logs to the directory specified by the `DIR_LOG` environment variable (defaults to `logs/`). +- **File Naming Format**: Log file names include: + - Application module (`APP_MODULE`) + - Logger name + - Hostname + - PID (Process ID) + + Example: `app_module_default_hostname_p12345.log` diff --git a/doc/skills/aloha_python/references/testing.md b/doc/skills/aloha_python/references/testing.md new file mode 100644 index 0000000..764ed36 --- /dev/null +++ b/doc/skills/aloha_python/references/testing.md @@ -0,0 +1,24 @@ +# Testing Utilities Module (`aloha.testing`) + +The `aloha.testing` subpackage provides base classes and helper utilities for writing unit tests and integration tests for API services. + +## 1. Unit Testing Base (`aloha.testing.unit`) + +Contains the base test case class for standard unit tests. + +### Key Classes + +- `UnitTestCase`: Extends `unittest.TestCase` and `abc.ABC`. + - Automatically loads the configured logger as `self.LOG`. + - Automatically loads the configuration dictionary as `self.config`. + +### Usage Example + +```python +from aloha.testing.unit import UnitTestCase + +class MyServiceTest(UnitTestCase): + def test_logic(self): + self.LOG.info("Running custom test...") + self.assertEqual(self.config.get("app_name"), "MyAlohaApp") +``` diff --git a/doc/skills/dev_env_cicd.md b/doc/skills/dev_env_cicd.md deleted file mode 100644 index 95c1188..0000000 --- a/doc/skills/dev_env_cicd.md +++ /dev/null @@ -1,72 +0,0 @@ -# Development Environment and CI/CD Management Skill - -This skill outlines the conventions and tools used within the `aloha-python` boilerplate for managing the development environment and CI/CD processes. It focuses on leveraging Docker and Docker Compose for a consistent and reproducible development workflow. - -## 1. Containerized Development Environment - -The `aloha-python` boilerplate provides a fully containerized development environment using Docker and Docker Compose. This ensures that all developers work with the same dependencies and configurations, minimizing "it works on my machine" issues. - -### Quick Start with `run-dev.sh` - -The primary script for managing the development environment is `./tool/cicd/run-dev.sh`. This script simplifies common Docker operations. - -**To launch the development environment:** - -```bash -./tool/cicd/run-dev.sh up -``` - -This command performs the following actions: -- **Port Availability Check**: Verifies that required ports (dynamically assigned based on user ID to avoid conflicts) are free. -- **Docker Image Build**: If the image doesn't exist, it builds it using `tool/dev-demo.Dockerfile` and `tool/cicd/docker-compose.app-demo.DEV.yml`. The build process includes installing Node.js (pnpm), Python with JupyterLab, project dependencies, and PostgreSQL client tools. -- **Container Start**: Starts the Docker container with volume mounts (e.g., `doc/`, `notebook/`, `src/` are mounted to `/root/app/doc`, `/root/app/notebook`, `/root/app/src` respectively) and port forwarding. - -**To enter the development container:** - -```bash -./tool/cicd/run-dev.sh enter -``` - -This command provides an interactive bash shell inside the running container, with the working directory set to `/root/app`. - -### `run-dev.sh` Commands - -The `run-dev.sh` script supports the following commands for environment management: - -| Command | Description | -| :------------------------------- | :------------------------------------------ | -| `./tool/cicd/run-dev.sh up` | Starts or creates the development environment. | -| `./tool/cicd/run-dev.sh restart` | Restarts the running container. | -| `./tool/cicd/run-dev.sh logs` | Views and follows container logs. | -| `./tool/cicd/run-dev.sh enter` | Accesses the container's bash shell. | -| `./tool/cicd/run-dev.sh down` | Stops and removes the container. | - -## 2. Dockerfile and Docker Compose Configuration - -- **`tool/dev-demo.Dockerfile`**: This Dockerfile defines the base image for the development container. It includes installations for `pnpm`, `jupyterlab`, Python dependencies, and database clients. -- **`tool/cicd/docker-compose.app-demo.DEV.yml`**: This Docker Compose file orchestrates the development environment. It specifies how the Docker image is built, environment variables (including `PYTHONPATH` for easy imports from `pkg/` and `src/`), exposed ports, and volume mounts to enable live code changes. - -## 3. Building Docker Images for Deployment - -The `tool/tool.sh` script provides utilities for building and managing Docker images for deployment. - -**To build a Docker image for your application:** - -```bash -source tool/tool.sh -build_image app_common latest tool/app-demo.Dockerfile -``` - -This command uses `tool/app-demo.Dockerfile` to build a production-ready Docker image for your application. - -## 4. Developing Documentation Locally - -To develop and preview documentation locally, the project uses MkDocs. - -**To serve the documentation locally:** - -```bash -mkdocs serve -f mkdocs.yml -a 0.0.0.0:3000 -``` - -This command starts a local web server, allowing you to view changes to your Markdown documentation in real-time. The main configuration files are `doc/mkdocs.yml` (for English) and `doc/mkdocs.zh.yml` (for Chinese). diff --git a/doc/skills/project_scaffolding.md b/doc/skills/project_scaffolding.md deleted file mode 100644 index 19a241e..0000000 --- a/doc/skills/project_scaffolding.md +++ /dev/null @@ -1,81 +0,0 @@ -# Project Scaffolding and Conventions Skill - -This skill provides guidelines for understanding and utilizing the `aloha-python` repository as a boilerplate for new Python projects. It outlines the project structure, the purpose of key directories, and how to leverage the `aloha` package within this framework. - -## 1. Project Structure Overview - -The `aloha-python` repository is organized into several top-level directories, each serving a specific purpose: - -- **`doc/`**: This directory contains all project documentation, including guides, API references, and these AI Agent Skills. When creating new documentation, it should be placed here, typically organized by language (e.g., `doc/en`, `doc/zh`). - -- **`pkg/`**: This directory stores the source code for the `aloha` Python package that is intended for publication to PyPI. It is the correct place to modify when the task is to work on this library itself. When using this repository as a boilerplate for a new application project, developers or agents should not include this directory unless they explicitly intend to create and publish a new package to PyPI. - -- **`src/`**: This directory is designed for application-specific code and tests that consume the `aloha` package. It serves as a boilerplate example (`app_common`) for how to structure a project using `aloha`. New projects based on this boilerplate should place their primary application logic, modules, and tests here. - -- **`notebook/`**: This directory is for Jupyter notebooks, which can be used for experimentation, data analysis, or interactive development related to the project. - -- **`tool/`**: This directory contains development and CI/CD related scripts and Docker assets. This includes scripts for setting up the development environment, building Docker images, and managing the project lifecycle. - -- **`AGENTS.md`**: Centralized file containing project-wide guidelines, rules, and style constraints shared across various AI agents. -- **`.agents/`**: Folder for Antigravity-specific customizations. The `skills/` subdirectory is symlinked to `doc/skills/` to load skills. -- **`.claude/`**: Folder for Claude Code-specific configurations. The `skills/` subdirectory is symlinked to `doc/skills/` to provide skills/instructions. -- **`.github/`**: Contains GitHub Actions workflows (`.github/workflows/`), Copilot instructions (`.github/copilot-instructions.md`, symlinked to `AGENTS.md`), and a symlink `.github/skills` to `../doc/skills`. - -## 2. Using the Repository as a Boilerplate - -To initiate a new project using `aloha-python` as a boilerplate, follow these steps: - -1. **Clone the Repository**: Begin by cloning the `aloha-python` repository to your local machine. -2. **Inspect `src/`**: Review the `src/` directory to understand the example application structure (`app_common`) and how it integrates with the `aloha` package. -3. **Develop Your Application**: Place your application-specific code within the `src/` directory, following the established patterns for modularity and `aloha` integration. -4. **Utilize `tool/cicd/`**: Leverage the scripts provided in `tool/cicd/` for managing your development environment, as detailed in the "Development Environment and CI/CD Management Skill". - -## 3. Key Conventions for Boilerplate Projects - -- **`src/` for Application Logic**: All primary application code, including API handlers, business logic, and utility modules, should reside within `src/`. The `src/main.py` script acts as a generic entry point for running Python modules within the `src/` directory. Your application's main function should be callable via `python3 src/main.py your_module.main`. -- **`pkg/` is not part of a new boilerplate app**: If the goal is to build a new application project from this repository, do not carry over `pkg/` unless the user specifically wants to create and publish a separate package. Application code should live in `src/` instead. -- **`resource/config/` for Configuration**: Application configuration files (e.g., `main.conf`, `deploy-DEV.conf`) should be placed under `src/resource/config/`. The `aloha` package's `aloha.config.paths` module handles the discovery and loading of these configuration files. For detailed information on HOCON configuration, refer to the "Configuration with HOCON" section in the `aloha_package_usage.md` skill. -- **Tests Placement**: All test-related code (including unit tests, integration tests, and test resources) must be placed inside the `src/` directory, typically organized under a `src/tests/` subdirectory. Test files should follow standard naming conventions such as `test_*.py`. -- **Executing Tests**: Tests should be run using `pytest` inside the containerized development environment: - 1. Launch and enter the development container: - ```bash - ./tool/cicd/run-dev.sh up - ./tool/cicd/run-dev.sh enter - ``` - 2. Run tests under the `src/` directory: - ```bash - pytest src/ - ``` - 3. To run tests with code coverage analysis: - ```bash - pytest --cov=src src/ - ``` - -By adhering to these conventions, AI agents can effectively understand, navigate, and contribute to projects built upon the `aloha-python` framework. - -## 4. Multi-Agent Setup and Scaffolding Script - -To facilitate cooperation and consistent behavior across multiple AI tools and agents (such as Antigravity/Gemini, Claude Code, and GitHub Copilot), the project uses a unified instruction layout. The following scaffolding script initializes this structure: - -```bash -# Create scaffolding directories -mkdir -pv .github/workflows doc/skills src/tests tool/cicd - -# Create agent config directories and symlink the centralized skills directory -mkdir -pv doc/skills .agents .claude \ -&& touch AGENTS.md \ -&& ln -sf AGENTS.md CLAUDE.md \ -&& ln -sf ../AGENTS.md .github/copilot-instructions.md \ -&& ln -sf ../doc/skills .agents/ \ -&& ln -sf ../doc/skills .claude/ \ -&& ln -sf ../doc/skills .github/ -``` - -### Key Components of the Multi-Agent Framework: -- **Centralized Agent Rules (`AGENTS.md`)**: - - `AGENTS.md` is the single source of truth for guidelines, code style constraints, and project rules. - - `CLAUDE.md` is symlinked to `AGENTS.md` for Claude Code. - - `.github/copilot-instructions.md` is symlinked to `../AGENTS.md` for GitHub Copilot. -- **Centralized Agent Skills (`doc/skills/`)**: - - Agent skills and instruction documents (like this one) are stored in the `doc/skills/` directory. - - Symlinks are created under `.agents/skills`, `.claude/skills`, and `.github/skills` pointing to `../doc/skills`. This ensures that all agents read the same skill instructions. From dcff8e802a9ba39b010551a3e44eddf6dd1a7857 Mon Sep 17 00:00:00 2001 From: Bibo Hao Date: Thu, 25 Jun 2026 20:58:09 +0800 Subject: [PATCH 3/4] update dep --- .gitignore | 4 + doc/.readthedocs.yaml | 1 - doc/requirements.txt | 41 -------- doc/skills/aloha_cicd/SKILL.md | 9 +- doc/skills/aloha_cicd/references/dev_env.md | 22 ++--- .../aloha_cicd/references/scaffolding.md | 1 + doc/skills/aloha_python/references/compile.md | 13 +-- doc/skills/aloha_python/references/config.md | 21 +++-- pkg/pyproject.toml | 93 +++++++++++++++++++ pkg/setup.py | 84 +++++------------ src/app-demo.Dockerfile | 4 +- src/dev-demo.Dockerfile | 8 +- src/pyproject.toml | 53 +++++++++++ src/requirements.txt | 26 ------ 14 files changed, 215 insertions(+), 165 deletions(-) delete mode 100644 doc/requirements.txt create mode 100644 pkg/pyproject.toml create mode 100644 src/pyproject.toml delete mode 100644 src/requirements.txt diff --git a/.gitignore b/.gitignore index 16cb9af..408210f 100644 --- a/.gitignore +++ b/.gitignore @@ -133,3 +133,7 @@ dmypy.json # Agents .codex + +# Generated README inside package folder +pkg/README.md + diff --git a/doc/.readthedocs.yaml b/doc/.readthedocs.yaml index 4f28b83..ecc4d79 100644 --- a/doc/.readthedocs.yaml +++ b/doc/.readthedocs.yaml @@ -22,7 +22,6 @@ mkdocs: python: install: - - requirements: ./doc/requirements.txt - method: pip path: ./pkg extra_requirements: diff --git a/doc/requirements.txt b/doc/requirements.txt deleted file mode 100644 index b0be5a1..0000000 --- a/doc/requirements.txt +++ /dev/null @@ -1,41 +0,0 @@ -# basic -attrdict3 -pyhocon -pycryptodome -packaging - -# build -Cython - -# service -requests -tornado -psutil -pyjwt - -# db -sqlalchemy -psycopg2-binary -pymysql -elasticsearch -pymongo -redis - -# stream -confluent_kafka - -# data -pandas - -# report -openpyxl -XlsxWriter - -# test -pytest-cov - -# docs -mkdocs -mkdocstrings[python] -markdown-include -mkdocs-material diff --git a/doc/skills/aloha_cicd/SKILL.md b/doc/skills/aloha_cicd/SKILL.md index 8b890a1..28dfcf6 100644 --- a/doc/skills/aloha_cicd/SKILL.md +++ b/doc/skills/aloha_cicd/SKILL.md @@ -11,7 +11,8 @@ This skill outlines the process for setting up boilerplate codebases, orchestrat ## 1. Project Scaffolding & Multi-Agent setup -To set up your project folders and ensure consistent AI assistant behaviors (across Antigravity, Claude Code, and Copilot), refer to the **[Project Scaffolding & Multi-Agent Setup reference](file:///home/haobibo/dev/dev-labnow/aloha-python/doc/skills/aloha_cicd/references/scaffolding.md)**. This guide covers: +To set up your project folders and ensure consistent AI assistant behaviors (across Antigravity, Claude Code, and Copilot), refer to the **[Project Scaffolding & Multi-Agent Setup reference](doc/skills/aloha_cicd/references/scaffolding.md)**. This guide covers: + - Core directory layout conventions (`src/`, `doc/`, `notebook/`, `tool/`). - Handing library isolation (excluding `pkg/` from new app template repositories). - Centralized rules (`AGENTS.md`) and centralized skills (`doc/skills/`) configuration via symlinks. @@ -20,7 +21,8 @@ To set up your project folders and ensure consistent AI assistant behaviors (acr ## 2. Local Development Environment -To start, run, and enter the development container with auto-allocated user ports and volume mappings, refer to the **[Local Development Container & Compose reference](file:///home/haobibo/dev/dev-labnow/aloha-python/doc/skills/aloha_cicd/references/dev_env.md)**. This guide covers: +To start, run, and enter the development container with auto-allocated user ports and volume mappings, refer to the **[Local Development Container & Compose reference](doc/skills/aloha_cicd/references/dev_env.md)**. This guide covers: + - Lifecycle management using `./tool/cicd/run-dev.sh` (`up`, `enter`, `logs`, `restart`, `down`). - User-specific dynamic port calculation (`PORT_APP` and `PORT_WEB`). - Mounting credentials and setting the `PYTHONPATH`. @@ -32,7 +34,7 @@ To start, run, and enter the development container with auto-allocated user port To compile your code into binary extension libraries (`.so` files) and package them into a lightweight container image: -1. **Production Dockerfile ([src/app-demo.Dockerfile](file:///home/haobibo/dev/dev-labnow/aloha-python/src/app-demo.Dockerfile))**: +1. **Production Dockerfile ([src/app-demo.Dockerfile](app-demo.Dockerfile))**: - Uses a two-stage build layout. - Converts normal Python `.py` files into binary dynamic libraries inside the builder stage (using `aloha compile`) if `ENABLE_CODE_BUILD="true"`. - Packs compiled assets into a clean minimal environment, exposing `PORT_SVC` (default 80) and launching via `aloha start`. @@ -54,6 +56,7 @@ Tests are executed using `pytest` inside the running dev container: ./tool/cicd/run-dev.sh enter ``` 2. Run your tests from the root directory: + ```bash # Run all unit and integration tests pytest src/ diff --git a/doc/skills/aloha_cicd/references/dev_env.md b/doc/skills/aloha_cicd/references/dev_env.md index 48b726f..0d0f089 100644 --- a/doc/skills/aloha_cicd/references/dev_env.md +++ b/doc/skills/aloha_cicd/references/dev_env.md @@ -10,13 +10,13 @@ To guarantee environment consistency, local development is fully containerized. ### Commands -| Command | Action | -| :--- | :--- | -| `./tool/cicd/run-dev.sh up` | Checks port availability and starts the development container in detached mode. | -| `./tool/cicd/run-dev.sh enter` | Spawns an interactive bash terminal inside the running container. | -| `./tool/cicd/run-dev.sh logs` | Tails and streams container logs to stdout. | -| `./tool/cicd/run-dev.sh restart` | Restarts the development container. | -| `./tool/cicd/run-dev.sh down` | Stops and removes the development container. | +| Command | Action | +| :------------------------------- | :------------------------------------------------------------------------------ | +| `./tool/cicd/run-dev.sh up` | Checks port availability and starts the development container in detached mode. | +| `./tool/cicd/run-dev.sh enter` | Spawns an interactive bash terminal inside the running container. | +| `./tool/cicd/run-dev.sh logs` | Tails and streams container logs to stdout. | +| `./tool/cicd/run-dev.sh restart` | Restarts the development container. | +| `./tool/cicd/run-dev.sh down` | Stops and removes the development container. | ### Configuration Details @@ -30,11 +30,11 @@ To guarantee environment consistency, local development is fully containerized. ## 2. Docker Compose & Dockerfiles Specs -- **Development Dockerfile ([src/dev-demo.Dockerfile](file:///home/haobibo/dev/dev-labnow/aloha-python/src/dev-demo.Dockerfile))**: +- **Development Dockerfile ([src/dev-demo.Dockerfile](src/dev-demo.Dockerfile))**: - Defines the base environment for development. - - Installs Node.js (`pnpm`), Python (`jupyterlab`), package requirements from `src/requirements.txt`, and PostgreSQL client CLI. -- **Development Compose ([tool/cicd/docker-compose.app-demo.DEV.yml](file:///home/haobibo/dev/dev-labnow/aloha-python/tool/cicd/docker-compose.app-demo.DEV.yml))**: + - Installs Node.js (`pnpm`), Python (`jupyterlab`), package requirements from [src/pyproject.toml](src/pyproject.toml), and PostgreSQL client CLI. +- **Development Compose ([tool/cicd/docker-compose.app-demo.DEV.yml](tool/cicd/docker-compose.app-demo.DEV.yml))**: - Configures the development service container, exposing computed ports, mounting workspace files, and setting environment variables. -- **Common Database Server ([tool/cicd/docker-compose.db.yml](file:///home/haobibo/dev/dev-labnow/aloha-python/tool/cicd/docker-compose.db.yml))**: +- **Common Database Server ([tool/cicd/docker-compose.db.yml](tool/cicd/docker-compose.db.yml))**: - Launches a shared PostgreSQL 17 database service (`db-postgres-common`) on port 5432. - Preloaded with database extensions: `pg_duckdb`, `pg_search`, `pg_cron`. diff --git a/doc/skills/aloha_cicd/references/scaffolding.md b/doc/skills/aloha_cicd/references/scaffolding.md index 32734a1..f30c2dc 100644 --- a/doc/skills/aloha_cicd/references/scaffolding.md +++ b/doc/skills/aloha_cicd/references/scaffolding.md @@ -37,5 +37,6 @@ mkdir -pv doc/skills .agents .claude \ ``` ### Integrated Workspace Layout + - **Centralized Rules (`AGENTS.md`)**: The single source of truth for code styling, conventions, and constraints. - **Centralized Skills (`doc/skills/`)**: Symlinked dynamically to `.agents/skills`, `.claude/skills`, and `.github/skills`. All agents automatically share the same custom instructions. diff --git a/doc/skills/aloha_python/references/compile.md b/doc/skills/aloha_python/references/compile.md index c4d5380..6111d23 100644 --- a/doc/skills/aloha_python/references/compile.md +++ b/doc/skills/aloha_python/references/compile.md @@ -11,6 +11,7 @@ aloha compile --base= --dist= --keep='' ``` ### Example + ```bash aloha compile --base=./src --dist=./dist_build --keep='main.py' ``` @@ -47,9 +48,9 @@ Under the hood, `aloha compile` executes the following steps: ## 4. CLI Parameter Reference -| Parameter | Type | Default | Description | -| :--- | :--- | :--- | :--- | -| `--base` | string | `./` | Root folder containing the Python source code to compile. | -| `--dist` | string | `build` | Destination directory where the final binary files and copied assets will be placed. | -| `--exclude` | string list | `()` | A list of file or folder path glob patterns to exclude from being copied or compiled. | -| `--keep` | string list | `()` | A list of specific `.py` files to keep as plain Python source code instead of compiling. | +| Parameter | Type | Default | Description | +| :---------- | :---------- | :------ | :--------------------------------------------------------------------------------------- | +| `--base` | string | `./` | Root folder containing the Python source code to compile. | +| `--dist` | string | `build` | Destination directory where the final binary files and copied assets will be placed. | +| `--exclude` | string list | `()` | A list of file or folder path glob patterns to exclude from being copied or compiled. | +| `--keep` | string list | `()` | A list of specific `.py` files to keep as plain Python source code instead of compiling. | diff --git a/doc/skills/aloha_python/references/config.md b/doc/skills/aloha_python/references/config.md index d10e688..3afe819 100644 --- a/doc/skills/aloha_python/references/config.md +++ b/doc/skills/aloha_python/references/config.md @@ -8,15 +8,15 @@ The `aloha.config` subpackage provides tools for discovering, loading, and parsi The configuration and startup behavior of an `aloha` application can be customized using the following environment variables: -| Environment Variable | Default Value | Description | -| :--- | :--- | :--- | -| `ENV_PROFILE` | `None` (undefined) | Specifies the running profile (e.g., `DEV`, `STG`, `PRD`). Determines the configuration entry file (`main-${ENV_PROFILE}.conf`). | -| `ENTRYPOINT` | `None` (undefined) | Specifies the default Python module entry point when running `aloha start`. The module must contain a `main()` function. | -| `APP_MODULE` | `default` | Identifies the application/module name. Mapped to configuration key `APP_MODULE` and used as a prefix for the log file names. | -| `DIR_LOG` | `logs` | The directory where log files are stored. | -| `DIR_RESOURCE` | `resource` (under CWD) | Root directory containing non-code resources (e.g. assets, static data). | -| `DIR_CONFIG` | `${DIR_RESOURCE}/config` | Directory where configuration files are located. | -| `FILES_CONFIG` | `None` (undefined) | Comma-separated list of configuration filenames to load (e.g., `db.conf,server.conf`). If defined, it overrides `ENV_PROFILE`. | +| Environment Variable | Default Value | Description | +| :------------------- | :----------------------- | :------------------------------------------------------------------------------------------------------------------------------- | +| `ENV_PROFILE` | `None` (undefined) | Specifies the running profile (e.g., `DEV`, `STG`, `PRD`). Determines the configuration entry file (`main-${ENV_PROFILE}.conf`). | +| `ENTRYPOINT` | `None` (undefined) | Specifies the default Python module entry point when running `aloha start`. The module must contain a `main()` function. | +| `APP_MODULE` | `default` | Identifies the application/module name. Mapped to configuration key `APP_MODULE` and used as a prefix for the log file names. | +| `DIR_LOG` | `logs` | The directory where log files are stored. | +| `DIR_RESOURCE` | `resource` (under CWD) | Root directory containing non-code resources (e.g. assets, static data). | +| `DIR_CONFIG` | `${DIR_RESOURCE}/config` | Directory where configuration files are located. | +| `FILES_CONFIG` | `None` (undefined) | Comma-separated list of configuration filenames to load (e.g., `db.conf,server.conf`). If defined, it overrides `ENV_PROFILE`. | --- @@ -25,6 +25,7 @@ The configuration and startup behavior of an `aloha` application can be customiz This module resolves paths for config directories, resource directories, and active configuration files. ### Key Functions + - `get_resource_dir(*args) -> str`: Resolves the absolute path to the resource directory. Relies on the `DIR_RESOURCE` environment variable. - `get_config_dir(*args) -> str`: Resolves the absolute path to the configuration directory. Relies on the `DIR_CONFIG` environment variable. - `get_config_files() -> list`: Determines which HOCON configuration files should be loaded. @@ -40,6 +41,7 @@ This module resolves paths for config directories, resource directories, and act This module uses the `pyhocon` library to parse configurations. ### Key Functions + - `load_config_from_hocon(config_file: str) -> dict`: Parses a single HOCON file into a plain ordered dictionary. - `load_config_from_hocon_files(config_files: list, base_dir: str) -> AttrDict`: Generates a string containing HOCON `include required("")` directives for each configuration file, parses it within the context of `base_dir`, and returns the merged configuration as an `AttrDict` object. This enables attribute-style dot access. @@ -50,6 +52,7 @@ This module uses the `pyhocon` library to parse configurations. The `aloha.settings` module exports a global, pre-instantiated singleton `SETTINGS` of class `Settings`, which manages lazy loading of config files. ### Usage Example + ```python from aloha.settings import SETTINGS diff --git a/pkg/pyproject.toml b/pkg/pyproject.toml new file mode 100644 index 0000000..5032152 --- /dev/null +++ b/pkg/pyproject.toml @@ -0,0 +1,93 @@ +[build-system] +requires = ["setuptools>=61.0.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "aloha" +dynamic = ["version"] +description = "Aloha - a versatile Python utility package for building services" +readme = "README.md" +requires-python = ">=3.10" +license = {text = "Apache-2.0"} +authors = [ + {name = "LabNow.ai", email = "postmaster@labnow.ai"} +] +classifiers = [ + "Intended Audience :: Developers", + "Intended Audience :: System Administrators", + "Intended Audience :: Science/Research", + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", +] +dependencies = [ + "attrdict3", + "pyhocon", + "pycryptodome", + "packaging", +] + +[project.urls] +Homepage = "https://github.com/LabNow.ai/aloha-python" +Source = "https://github.com/LabNow-ai/aloha-python" +"CI Pipeline" = "https://github.com/LabNow-ai/aloha-python/actions" +Documentation = "https://aloha-python.readthedocs.io/" + +[project.scripts] +aloha = "aloha.script.base:main" + +[project.optional-dependencies] +build = ["Cython"] +service = ["psutil", "pyjwt", "fastapi", "httpx", "uvicorn"] +db = [ + "sqlalchemy", + "psycopg[binary]", + "pymysql", + "elasticsearch", + "pymongo", + "redis", + "duckdb", + "duckdb-engine", + "oracledb", +] +stream = ["confluent_kafka"] +data = ["pandas", "lxml"] +report = ["openpyxl", "XlsxWriter"] +test = ["pytest-cov"] +docs = ["mkdocs", "mkdocstrings[python]", "markdown-include", "mkdocs-material"] +all = [ + "Cython", + "psutil", + "pyjwt", + "fastapi", + "httpx", + "uvicorn", + "sqlalchemy", + "psycopg[binary]", + "pymysql", + "elasticsearch", + "pymongo", + "redis", + "duckdb", + "duckdb-engine", + "oracledb", + "confluent_kafka", + "pandas", + "lxml", + "openpyxl", + "XlsxWriter", + "pytest-cov", + "mkdocs", + "mkdocstrings[python]", + "markdown-include", + "mkdocs-material" +] + +[tool.setuptools] +include-package-data = true +zip-safe = false + +[tool.setuptools.packages.find] +where = ["."] + +[tool.setuptools.dynamic] +version = {attr = "aloha._version.__version__"} diff --git a/pkg/setup.py b/pkg/setup.py index 7987507..650d860 100644 --- a/pkg/setup.py +++ b/pkg/setup.py @@ -1,70 +1,28 @@ +import os +import shutil from datetime import datetime -from setuptools import find_packages, setup +from setuptools import setup -with open("README.md", "r") as fh: - long_description = fh.read() +# Get directory containing this setup.py +base_dir = os.path.dirname(os.path.abspath(__file__)) -_now = datetime.now() -_version = "%s.%02d%02d.%02d%02d" % (_now.year, _now.month, _now.day, _now.hour, _now.minute) +# 1. Resolve README.md presence +# In Python package builds, README.md is required to be in the same directory. +# If it is missing (e.g., built locally without github workflow cp command), we copy it from the root directory. +readme_path = os.path.join(base_dir, "README.md") +root_readme = os.path.join(base_dir, "..", "README.md") +if not os.path.exists(readme_path) and os.path.exists(root_readme): + shutil.copyfile(root_readme, readme_path) -with open("./aloha/_version.py", "wt") as fp: - fp.write('__version__ = "%s"\n' % _version) +# 2. Dynamic Version Generation +# Writes the version to aloha/_version.py using the current timestamp. +_t = datetime.now() +_version = "%s.%02d%02d.%02d%02d" % (_t.year, _t.month, _t.day, _t.hour, _t.minute) -dict_extra_requires = { - "build": ["Cython"], - "service": ["psutil", "pyjwt", "fastapi", "httpx", "uvicorn"], - "db": [ - "sqlalchemy", - "psycopg[binary]", - "pymysql", - "elasticsearch", - "pymongo", - "redis", - "duckdb", - "duckdb-engine", - "oracledb", - ], - "stream": ["confluent_kafka"], - "data": ["pandas", "lxml"], - "report": ["openpyxl", "XlsxWriter"], - "test": ["pytest-cov"], - "docs": ["mkdocs", "mkdocstrings[python]", "markdown-include", "mkdocs-material"], -} +version_file_path = os.path.join(base_dir, "aloha", "_version.py") +with open(version_file_path, "wt") as fp: + fp.write('__version__ = "%s"\n' % _version) -setup( - name="aloha", - version=_version, - author="LabNow.ai", - author_email="postmaster@labnow.ai", - license="Apache-2.0", - url="https://github.com/LabNow.ai/aloha-python", - project_urls={ - "Source": "https://github.com/LabNow-ai/aloha-python", - "CI Pipeline": "https://github.com/LabNow-ai/aloha-python/actions", - "Documentation": "https://aloha-python.readthedocs.io/", - }, - packages=find_packages(where=".", exclude=("app_common*",)), - include_package_data=True, - package_data={}, - platforms="Linux, Mac OS X, Windows", - zip_safe=False, - install_requires=["attrdict3", "pyhocon", "pycryptodome", "packaging"], - extras_require={ - **dict_extra_requires, - "all": sorted(y for x in dict_extra_requires.values() for y in x), - }, - python_requires=">=3.10", - entry_points={"console_scripts": ["aloha = aloha.script.base:main"]}, - data_files=[], - description="Aloha - a versatile Python utility package for building services", - long_description=long_description, - long_description_content_type="text/markdown", - classifiers=[ - "Intended Audience :: Developers", - "Intended Audience :: System Administrators", - "Intended Audience :: Science/Research", - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - ], -) +# 3. Trigger setup (reads configuration from pyproject.toml) +setup() diff --git a/src/app-demo.Dockerfile b/src/app-demo.Dockerfile index 8313686..7c474c4 100644 --- a/src/app-demo.Dockerfile +++ b/src/app-demo.Dockerfile @@ -41,8 +41,8 @@ ENV ENTRYPOINT="app_common.debug" RUN set -eux && pwd && ls -alh \ && source /opt/utils/script-localize.sh ${PROFILE_LOCALIZE} \ - && pip install -U --no-cache-dir pip \ - && ( [ -f ./requirements.txt ] && pip install -U --no-cache-dir -r ./requirements.txt || true ) && pip list \ + && pip install -U --no-cache-dir pip uv \ + && ( [ -f ./pyproject.toml ] && uv pip install --system -r ./pyproject.toml || true ) && pip list \ && rm -rf ~/.cache && ls -al && printenv | sort VOLUME ${DIR_APP}/logs diff --git a/src/dev-demo.Dockerfile b/src/dev-demo.Dockerfile index babf62f..3d06ee4 100644 --- a/src/dev-demo.Dockerfile +++ b/src/dev-demo.Dockerfile @@ -8,7 +8,7 @@ FROM ${BASE_NAMESPACE:+$BASE_NAMESPACE/}${BASE_IMG} AS dev ARG PROFILE_LOCALIZE -COPY src/requirements.txt /tmp/ +COPY src/pyproject.toml /tmp/ USER root RUN set -eux && pwd && ls -alh \ @@ -16,9 +16,11 @@ RUN set -eux && pwd && ls -alh \ # ----------- handle frontend matters ----------- && npm install -g pnpm \ # ----------- handle backend matters ------------ - && pip install -U --no-cache-dir pip jupyterlab \ - && pip install -U --no-cache-dir -r /tmp/requirements.txt \ + && pip install -U --no-cache-dir pip jupyterlab uv \ + && uv pip install --system -r /tmp/pyproject.toml \ # ----------- install db client to connect db via terminal ------------ && source /opt/utils/script-setup-db-clients.sh && setup_postgresql_client 17 \ # ----------- clean up ----------- && source /opt/utils/script-utils.sh && list_installed_packages && install__clean + +CMD ["tail", "-f", "/dev/null"] diff --git a/src/pyproject.toml b/src/pyproject.toml new file mode 100644 index 0000000..016123b --- /dev/null +++ b/src/pyproject.toml @@ -0,0 +1,53 @@ +[build-system] +requires = ["setuptools>=61.0.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "app-common" +version = "0.1.0" +description = "Aloha Application Common boilerplate" +requires-python = ">=3.10" +dependencies = [ + # Core aloha dependencies + "attrdict3", + "pyhocon", + "pycryptodome", + "packaging", + + # Extras: build + "Cython", + + # Extras: service + "psutil", + "pyjwt", + "fastapi", + "httpx", + "uvicorn", + + # Extras: db + "sqlalchemy", + "psycopg[binary]", + "pymysql", + "elasticsearch", + "pymongo", + "redis", + "duckdb", + "duckdb-engine", + "oracledb", + + # Extras: stream + "confluent_kafka", + + # Extras: data & report + "pandas", + "lxml", + "openpyxl", + "XlsxWriter", + + # Extras: test & docs + "pytest-cov", + "mkdocs", + "mkdocstrings[python]", + "markdown-include", + "mkdocs-material", +] diff --git a/src/requirements.txt b/src/requirements.txt deleted file mode 100644 index 67b4f89..0000000 --- a/src/requirements.txt +++ /dev/null @@ -1,26 +0,0 @@ -# aloha[all] -attrdict3 -pyhocon -pycryptodome -packaging -Cython -psutil -pyjwt -fastapi -httpx -uvicorn -sqlalchemy -psycopg[binary] -pymysql -elasticsearch -pymongo -redis -confluent_kafka -pandas -openpyxl -XlsxWriter -pytest-cov -mkdocs -mkdocstrings[python] -markdown-include -mkdocs-material From a3b0f8802d82ec434a8634e7ae480cd0f054e692 Mon Sep 17 00:00:00 2001 From: Bibo Hao Date: Thu, 25 Jun 2026 21:08:34 +0800 Subject: [PATCH 4/4] update README --- LICENSE | 366 ++++++++++++++++++++++++------------------------ README.md | 84 +++++++++-- doc/zh/index.md | 2 +- 3 files changed, 256 insertions(+), 196 deletions(-) diff --git a/LICENSE b/LICENSE index 261eeb9..343c8f5 100644 --- a/LICENSE +++ b/LICENSE @@ -2,180 +2,180 @@ Version 2.0, January 2004 http://www.apache.org/licenses/ - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" @@ -186,16 +186,16 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright [yyyy] [name of copyright owner] +Copyright 2024 LabNow.ai - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/README.md b/README.md index c21c755..28c775f 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,5 @@ # Aloha -## What is it? - -`aloha` is a versatile Python utility package for building microservices. - [![License](https://img.shields.io/github/license/LabNow-ai/aloha-python)](https://github.com/LabNow-ai/aloha-python/blob/main/LICENSE) [![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LabNow-ai/aloha-python/build.yml?branch=main)](https://github.com/LabNow-ai/aloha-python/actions) [![PyPI version](https://img.shields.io/pypi/v/aloha)](https://pypi.python.org/pypi/aloha/) @@ -11,6 +7,10 @@ [![Code Activity](https://img.shields.io/github/commit-activity/m/LabNow-ai/aloha-python)](https://github.com/LabNow-ai/aloha-python/pulse) [![Recent Code Update](https://img.shields.io/github/last-commit/LabNow-ai/aloha-python.svg)](https://github.com/LabNow-ai/aloha-python/stargazers) +`aloha-python` is a modern project template (boilerplate) and a versatile utility library (`aloha` package) designed to build robust, containerized microservices in Python. + +--- + Please generously STAR★ our project or donate to us! [![GitHub Starts](https://img.shields.io/github/stars/LabNow-ai/aloha-python.svg?label=Stars&style=social)](https://github.com/LabNow-ai/aloha-python/stargazers) @@ -18,18 +18,78 @@ Please generously STAR★ our project or donate to us! - To contribute or talk to a human: [![Open an Issue on GitHub](https://img.shields.io/github/issues/LabNow-ai/aloha-python)](https://github.com/LabNow-ai/aloha-python/issues) [![Join the Discord Chat](https://img.shields.io/badge/Discuss_on-Discord-green)](https://discord.gg/kHUzgQxgbJ) [![Join the Gitter Chat](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/LabNow-ai/) +--- + +## 🚀 Key Features + +- **Configuration Management (`aloha.config`)**: Lazy-loaded settings (`SETTINGS`) using HOCON (Human-Optimized Config Object Notation), supporting environment profile overrides (`ENV_PROFILE` / `FILES_CONFIG`) and environment variable injection. +- **Concurrent-Safe Logging (`aloha.logger`)**: Multi-process safe daily rotating log file handler, console output, and automatic log paths configuration. +- **Database Operators (`aloha.db`)**: Pre-built SQLAlchemy-backed connections for PostgreSQL, MySQL, SQLite, DuckDB, MongoDB, Redis, Elasticsearch, and Kafka, with password resolution via a secure `PasswordVault` wrapper. +- **Encryption & Utilities (`aloha.encrypt`)**: Fast helpers for AES (ECB/CBC) encryption, RSA asymmetric key-pair generation/signatures, JWT encoding/decoding, and Base62 hashing. +- **Testing Harness (`aloha.testing`)**: Extended `UnitTestCase` and integration `ServiceTestCase` structures for testing HTTP endpoints. +- **Binary Code Protection (`aloha compile`)**: Utility compiler using Cython to package python source files (`.py`) into platform-native compiled dynamic libraries (`.so`/`.pyd`), protecting intellectual property. + +--- + +## 📁 Repository Layout + +- **[`src/`](src)**: Application-specific codebase, configuration files (`src/resource/config/`), and unit/integration tests (`src/tests/`). +- **[`pkg/`](pkg)**: The core `aloha` utility library source code. +- **[`tool/`](tool)**: Local CI/CD files and setup helpers (e.g. docker-compose configurations, lifecycle scripts). +- **[`doc/`](doc)**: Documentation files and AI Agent Skills. +- **[`notebook/`](notebook)**: Jupyter notebooks for interactive analysis. + +--- + +## 🛠️ Local Development Quick Start + +Local development is fully containerized using Docker & Docker Compose to maintain environment consistency. + +### 1. Launch Dev Container -## Getting started +Run the lifecycle helper script: -Refer to 📚 [Document](https://aloha-python.readthedocs.io/) & [中文文档](https://aloha-python.readthedocs.io/zh-cn/main/) for detailed introduction. +```bash +# Check port availability and spin up the development container +./tool/cicd/run-dev.sh up -```shell -pip install aloha[all] +# Open an interactive terminal inside the container +./tool/cicd/run-dev.sh enter ``` -And then: +_Note: Development ports (`PORT_APP` and `PORT_WEB`) are dynamically computed based on your User ID (`UID`) to avoid conflicts on shared servers._ -```python -from aloha.logger import LOG -from aloha.settings import SETTINGS as S +### 2. Run Tests + +Inside the running container terminal: + +```bash +# Execute pytest suite +pytest src/ + +# Run tests with code coverage report +pytest --cov=src src/ +``` + +### 3. Production Packaging + +To build a production-ready Docker image with optional binary compilation enabled: + +```bash +source tool/tool.sh +build_image app_common latest src/app-demo.Dockerfile ``` + +--- + +## ✍️ Coding Guidelines + +When developing in this project, variables should place their **type or primary characteristics/role prefix first**: + +- _Correct_: `name_service`, `port_service`, `svc_ingress`, `cfg_postgres`. +- _Incorrect_: `service_name`, `service_port`, `ingress_service`, `postgres_config`. + +For details on local setups, CI/CD specifications, and sub-module APIs, inspect our agent instruction files: + +- **[Aloha Python Skills Guide](doc/skills/aloha_python/SKILL.md)** +- **[Aloha CI/CD & Scaffolding Guide](doc/skills/aloha_cicd/SKILL.md)** diff --git a/doc/zh/index.md b/doc/zh/index.md index 18f57c8..97019da 100644 --- a/doc/zh/index.md +++ b/doc/zh/index.md @@ -1,6 +1,6 @@ # 介绍 -Aloha！感谢你关注这个 Python 包。 +Aloha！感谢关注这个 Python 包。 [![License](https://img.shields.io/github/license/LabNow-ai/aloha-python)](https://github.com/LabNow-ai/aloha-python/blob/main/LICENSE) [![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LabNow-ai/aloha-python/build.yml?branch=main)](https://github.com/LabNow-ai/aloha-python/actions)