diff --git a/doc/en/README-develop.md b/doc/en/README-develop.md index 3ee78e5..b36a797 100644 --- a/doc/en/README-develop.md +++ b/doc/en/README-develop.md @@ -1,9 +1,47 @@ # Development Docs +## Quick Start Development Environment + +The project provides a containerized development environment using Docker and Docker Compose. It sets up a complete development workspace with all necessary dependencies pre-installed. + +### Launch the Development Environment + +Run this command in your terminal: + +```bash +./tool/cicd/run-dev.sh up +``` + +This will build the Docker image and start the container. Your local code directories are mounted into the container, enabling live development. + +### Enter the Development Container + +Once the environment is running, execute: + +```bash +./tool/cicd/run-dev.sh enter +``` + +You will be attached to the running container with a bash shell. + +### Manage the Environment + +The `run-dev.sh` script provides several commands to manage your development environment: + +| Command | Description | +| -------------------------------- | ------------------------------------------- | +| `./tool/cicd/run-dev.sh up` | Start or create the development environment | +| `./tool/cicd/run-dev.sh restart` | Restart the running container | +| `./tool/cicd/run-dev.sh logs` | View and follow container logs | +| `./tool/cicd/run-dev.sh enter` | Access the container's bash shell | +| `./tool/cicd/run-dev.sh down` | Stop and remove the container | + ## Live debug source code with Docker +If you prefer to run a single container for debugging: + ```bash -# First, cd to project root (which includes `src`), then run: +# First, cd to project root, then run: docker run -it \ -v $(pwd):/root/app/ \ -w /root/app/src \ @@ -11,14 +49,14 @@ docker run -it \ -p 8080:80 \ quay.io/labnow/base:latest bash -python -m aloha.script.start app_common.debug +python3 main.py app_common.debug ``` ## Build Docker image ```bash source tool/tool.sh -build_image app_common latest tool/app.Dockerfile +build_image app_common latest tool/app-demo.Dockerfile ``` ## Develop docs diff --git a/doc/en/README-get-start.md b/doc/en/README-get-start.md index c82617b..168968d 100644 --- a/doc/en/README-get-start.md +++ b/doc/en/README-get-start.md @@ -8,7 +8,7 @@ pip install aloha[all] ## Step 2. Use this repository as a boilerplate -The `app/` folder in this repository is a boilerplate/template project built on top of `aloha`. +This repository serves as a boilerplate/template project built on top of `aloha`. It gives you a ready-to-use application layout, development scripts, and containerized tooling so you can start building instead of assembling the project skeleton yourself. ### What this template gives you @@ -21,7 +21,7 @@ It gives you a ready-to-use application layout, development scripts, and contain ### Recommended workflow 1. Clone this repository. -2. Open the `app/` directory to inspect the starter application structure. +2. Inspect the starter application structure. 3. Use the scripts under `tool/cicd/` to start the development container when you want a reproducible environment. 4. Put your own application code in the template structure and grow from there. @@ -40,10 +40,77 @@ The `up` command creates or starts the development container. The `enter` comman The template is organized around a few common folders: -- `app/`: application code and entry points -- `src/`: the `aloha` library source code -- `doc/`: documentation source files -- `notebook/`: Jupyter notebooks for experimentation -- `tool/`: scripts and Docker assets for development and CI/CD +- `doc/`: Documentation source files. You can place your project's documentation here. +- `src/`: Application code and entry points. This is where your business logic goes. It contains a demo application (`app_common`) showing how to use `aloha`. +- `tool/`: Scripts and Docker assets for development and CI/CD. -[:octicons-mark-github-16: Go to Template Project](https://github.com/LabNow-ai/aloha-python/tree/main/app){ .md-button } +### How to use `aloha` in your project + +The `src/` directory contains a demo application that demonstrates how to use the `aloha` package. Here is a brief overview of how to import and use `aloha` for regular Python project development: + +1. **Import `aloha` modules**: Import the necessary `aloha` modules for your application logic. For example, `aloha.logger` for logging, `aloha.config` for configuration, or `aloha.db` for database interactions. + +2. **Configure your application**: Define your application configuration in `src/resource/config/main.conf`. + +3. **Implement your application logic**: Write your Python code using the imported `aloha` modules. + +4. **Run your application**: You can run your application using the provided `main.py` script: + +```bash +python3 src/main.py your_module.main +``` + +This command tells `src/main.py` to find and execute the `main()` function within your specified module (e.g., `your_module.main`). + +## Configuration with HOCON + +`aloha` uses the HOCON (Human-Optimized Config Object Notation) format for its configuration files. This allows for hierarchical, modular, and human-readable configurations. + +### Configuration File Location + +By default, `aloha` looks for configuration files in the `src/resource/config/` directory. The primary configuration file is `main.conf`. + +### Modular Configuration + +HOCON supports including other configuration files, which is useful for modularizing your settings (e.g., separating development, staging, and production configurations). + +**Example `main.conf`:** + +```hocon +include "deploy-DEV.conf" + +app_name = "MyAlohaApp" + +server { + host = "0.0.0.0" + port = 8080 +} + +database { + type = "postgresql" + connection_string = "${?DB_CONNECTION_STRING}" # Environment variable override +} +``` + +In this example: +- `include "deploy-DEV.conf"` brings in settings from another file, allowing for environment-specific overrides. +- `app_name` and `server` define application-specific settings. +- `database.connection_string = "${?DB_CONNECTION_STRING}"` demonstrates how to use environment variables to override configuration values, making it flexible for different deployment environments. + +### Accessing Configuration in Code + +You can access configuration values in your Python code via `aloha.settings.SETTINGS.config`: + +```python +from aloha.settings import SETTINGS + +app_name = SETTINGS.config.get("app_name") +server_port = SETTINGS.config.get("server.port") + +print(f"Application Name: {app_name}") +print(f"Server Port: {server_port}") +``` + +This approach ensures that your application remains configurable and adaptable to various deployment scenarios. + +[:octicons-mark-github-16: Go to Template Project](https://github.com/LabNow-ai/aloha-python/tree/main/src){ .md-button } diff --git a/doc/skills/aloha_package_usage.md b/doc/skills/aloha_package_usage.md new file mode 100644 index 0000000..c6d5709 --- /dev/null +++ b/doc/skills/aloha_package_usage.md @@ -0,0 +1,102 @@ +# Aloha Package Usage and API Development Skill + +This skill provides guidance on how to effectively use the `aloha` Python package for developing applications within the `aloha-python` boilerplate structure. + +## 1. Importing and Using the `aloha` Package + +The `aloha` package is designed to be easily integrated into your Python projects. Once installed (e.g., `pip install aloha[all]`), you can import its modules as needed. + +### Example: Basic `aloha` Module Usage + +To use `aloha` modules, you simply import them and use their functionalities. For example, using `aloha.logger`: + +```python +from aloha.logger import LOG + +def my_function(): + LOG.info("This is an informational message from aloha logger.") + return "Function executed successfully" +``` + +In this example: +- `aloha.logger.LOG` provides a pre-configured logger instance. + +## 2. Application Configuration with HOCON + +`aloha` applications are configured using HOCON (Human-Optimized Config Object Notation) files, typically located in `src/resource/config/`. The `aloha.config.paths` module helps in discovering these configuration files. + +### Configuration File Location and Structure + +By default, `aloha` looks for configuration files in the `src/resource/config/` directory. The primary configuration file is `main.conf`. + +HOCON allows for hierarchical, modular, and human-readable configurations. You can include other configuration files, which is useful for modularizing your settings (e.g., separating development, staging, and production configurations). + +**Example `main.conf`:** + +```hocon +include "deploy-DEV.conf" + +app_name = "MyAlohaApp" + +server { + host = "0.0.0.0" + port = 8080 +} + +database { + type = "postgresql" + connection_string = "${?DB_CONNECTION_STRING}" # Environment variable override +} +``` + +In this example: +- `include "deploy-DEV.conf"` brings in settings from another file, allowing for environment-specific overrides. +- `app_name` and `server` define application-specific settings. +- `database.connection_string = "${?DB_CONNECTION_STRING}"` demonstrates how to use environment variables to override configuration values, making it flexible for different deployment environments. + +### Accessing Configuration in Code + +You can access configuration values in your Python code via `aloha.settings.SETTINGS.config`: + +```python +from aloha.settings import SETTINGS + +config = SETTINGS.config +print(f"Application Name: {config.get("app_name")}") +print(f"Server Port: {config.get("server.port")}") +``` + +This approach ensures that your application remains configurable and adaptable to various deployment scenarios. + +## 3. Running Your `aloha`-based Application + +To run your `aloha`-based application, you can use the `src/main.py` script, which acts as a generic module runner: + +### Example: Application Entry Point + +Your application's main entry point should be a function (e.g., `main()`) within a module in your `src/` directory. For example, in `src/my_app/main.py`: + +```python +def main(): + # Your application's startup logic here + print("My Aloha application is starting...") + # Integrate with FastAPI or other frameworks here +``` + +### Running the Application + +To execute your application, use the `src/main.py` script: + +```bash +python3 src/main.py my_app.main +``` + +This command tells `src/main.py` to find and execute the `main()` function within your specified module (e.g., `my_app.main`). + +## 4. Advanced Usage and Best Practices + +- **Configuration Management**: Leverage `aloha.config.paths` and HOCON files for environment-specific configurations (e.g., `deploy-DEV.conf`). This allows for modular and flexible management of settings across different environments. +- **HOCON Configuration Details**: For a comprehensive understanding of HOCON configuration, refer to the "Configuration with HOCON" section in the `README-get-start.md` documentation. +- **Database Integration**: Utilize `aloha.db` modules (e.g., `aloha.db.postgres`) for seamless database interactions, as demonstrated in `src/app_common/api/api_common_query_postgres.py`. +- **Logging**: Use `aloha.logger.LOG` for consistent and configurable logging across your application. +- **Testing**: Employ `aloha.testing` utilities for unit and integration tests. diff --git a/doc/skills/dev_env_cicd.md b/doc/skills/dev_env_cicd.md new file mode 100644 index 0000000..95c1188 --- /dev/null +++ b/doc/skills/dev_env_cicd.md @@ -0,0 +1,72 @@ +# Development Environment and CI/CD Management Skill + +This skill outlines the conventions and tools used within the `aloha-python` boilerplate for managing the development environment and CI/CD processes. It focuses on leveraging Docker and Docker Compose for a consistent and reproducible development workflow. + +## 1. Containerized Development Environment + +The `aloha-python` boilerplate provides a fully containerized development environment using Docker and Docker Compose. This ensures that all developers work with the same dependencies and configurations, minimizing "it works on my machine" issues. + +### Quick Start with `run-dev.sh` + +The primary script for managing the development environment is `./tool/cicd/run-dev.sh`. This script simplifies common Docker operations. + +**To launch the development environment:** + +```bash +./tool/cicd/run-dev.sh up +``` + +This command performs the following actions: +- **Port Availability Check**: Verifies that required ports (dynamically assigned based on user ID to avoid conflicts) are free. +- **Docker Image Build**: If the image doesn't exist, it builds it using `tool/dev-demo.Dockerfile` and `tool/cicd/docker-compose.app-demo.DEV.yml`. The build process includes installing Node.js (pnpm), Python with JupyterLab, project dependencies, and PostgreSQL client tools. +- **Container Start**: Starts the Docker container with volume mounts (e.g., `doc/`, `notebook/`, `src/` are mounted to `/root/app/doc`, `/root/app/notebook`, `/root/app/src` respectively) and port forwarding. + +**To enter the development container:** + +```bash +./tool/cicd/run-dev.sh enter +``` + +This command provides an interactive bash shell inside the running container, with the working directory set to `/root/app`. + +### `run-dev.sh` Commands + +The `run-dev.sh` script supports the following commands for environment management: + +| Command | Description | +| :------------------------------- | :------------------------------------------ | +| `./tool/cicd/run-dev.sh up` | Starts or creates the development environment. | +| `./tool/cicd/run-dev.sh restart` | Restarts the running container. | +| `./tool/cicd/run-dev.sh logs` | Views and follows container logs. | +| `./tool/cicd/run-dev.sh enter` | Accesses the container's bash shell. | +| `./tool/cicd/run-dev.sh down` | Stops and removes the container. | + +## 2. Dockerfile and Docker Compose Configuration + +- **`tool/dev-demo.Dockerfile`**: This Dockerfile defines the base image for the development container. It includes installations for `pnpm`, `jupyterlab`, Python dependencies, and database clients. +- **`tool/cicd/docker-compose.app-demo.DEV.yml`**: This Docker Compose file orchestrates the development environment. It specifies how the Docker image is built, environment variables (including `PYTHONPATH` for easy imports from `pkg/` and `src/`), exposed ports, and volume mounts to enable live code changes. + +## 3. Building Docker Images for Deployment + +The `tool/tool.sh` script provides utilities for building and managing Docker images for deployment. + +**To build a Docker image for your application:** + +```bash +source tool/tool.sh +build_image app_common latest tool/app-demo.Dockerfile +``` + +This command uses `tool/app-demo.Dockerfile` to build a production-ready Docker image for your application. + +## 4. Developing Documentation Locally + +To develop and preview documentation locally, the project uses MkDocs. + +**To serve the documentation locally:** + +```bash +mkdocs serve -f mkdocs.yml -a 0.0.0.0:3000 +``` + +This command starts a local web server, allowing you to view changes to your Markdown documentation in real-time. The main configuration files are `doc/mkdocs.yml` (for English) and `doc/mkdocs.zh.yml` (for Chinese). diff --git a/doc/skills/project_scaffolding.md b/doc/skills/project_scaffolding.md new file mode 100644 index 0000000..72768ed --- /dev/null +++ b/doc/skills/project_scaffolding.md @@ -0,0 +1,34 @@ +# Project Scaffolding and Conventions Skill + +This skill provides guidelines for understanding and utilizing the `aloha-python` repository as a boilerplate for new Python projects. It outlines the project structure, the purpose of key directories, and how to leverage the `aloha` package within this framework. + +## 1. Project Structure Overview + +The `aloha-python` repository is organized into several top-level directories, each serving a specific purpose: + +- **`doc/`**: This directory contains all project documentation, including guides, API references, and these AI Agent Skills. When creating new documentation, it should be placed here, typically organized by language (e.g., `doc/en`, `doc/zh`). + +- **`pkg/`**: This directory stores the source code for the `aloha` Python package that is intended for publication to PyPI. It is the correct place to modify when the task is to work on this library itself. When using this repository as a boilerplate for a new application project, developers or agents should not include this directory unless they explicitly intend to create and publish a new package to PyPI. + +- **`src/`**: This directory is designed for the application-specific code that consumes the `aloha` package. It serves as a boilerplate example (`app_common`) for how to structure a project using `aloha`. New projects based on this boilerplate should place their primary application logic and modules here. + +- **`notebook/`**: This directory is for Jupyter notebooks, which can be used for experimentation, data analysis, or interactive development related to the project. + +- **`tool/`**: This directory contains development and CI/CD related scripts and Docker assets. This includes scripts for setting up the development environment, building Docker images, and managing the project lifecycle. + +## 2. Using the Repository as a Boilerplate + +To initiate a new project using `aloha-python` as a boilerplate, follow these steps: + +1. **Clone the Repository**: Begin by cloning the `aloha-python` repository to your local machine. +2. **Inspect `src/`**: Review the `src/` directory to understand the example application structure (`app_common`) and how it integrates with the `aloha` package. +3. **Develop Your Application**: Place your application-specific code within the `src/` directory, following the established patterns for modularity and `aloha` integration. +4. **Utilize `tool/cicd/`**: Leverage the scripts provided in `tool/cicd/` for managing your development environment, as detailed in the "Development Environment and CI/CD Management Skill". + +## 3. Key Conventions for Boilerplate Projects + +- **`src/` for Application Logic**: All primary application code, including API handlers, business logic, and utility modules, should reside within `src/`. The `src/main.py` script acts as a generic entry point for running Python modules within the `src/` directory. Your application's main function should be callable via `python3 src/main.py your_module.main`. +- **`pkg/` is not part of a new boilerplate app**: If the goal is to build a new application project from this repository, do not carry over `pkg/` unless the user specifically wants to create and publish a separate package. Application code should live in `src/` instead. +- **`resource/config/` for Configuration**: Application configuration files (e.g., `main.conf`, `deploy-DEV.conf`) should be placed under `src/resource/config/`. The `aloha` package's `aloha.config.paths` module handles the discovery and loading of these configuration files. For detailed information on HOCON configuration, refer to the "Configuration with HOCON" section in the `aloha_package_usage.md` skill. + +By adhering to these conventions, AI agents can effectively understand, navigate, and contribute to projects built upon the `aloha-python` framework. diff --git a/doc/zh/README-develop.md b/doc/zh/README-develop.md index 4d3fba1..8889400 100644 --- a/doc/zh/README-develop.md +++ b/doc/zh/README-develop.md @@ -1,9 +1,47 @@ # 开发文档 -## 使用 Docker 对源码进行实时调试 +## 快速启动开发环境 + +本项目提供了一个使用 Docker 和 Docker Compose 的容器化开发环境。它设置了一个完整的开发工作区，预装了所有必要的依赖项。 + +### 启动开发环境 + +在你的终端中运行此命令： + +```bash +./tool/cicd/run-dev.sh up +``` + +这将构建 Docker 镜像并启动容器。你的本地代码目录将挂载到容器中，实现实时开发。 + +### 进入开发容器 + +环境运行后，执行： + +```bash +./tool/cicd/run-dev.sh enter +``` + +你将连接到正在运行的容器，并获得一个 bash shell。 + +### 管理环境 + +`run-dev.sh` 脚本提供了几个命令来管理你的开发环境： + +| 命令 | 描述 | +| -------------------------------- | ------------------------------------------- | +| `./tool/cicd/run-dev.sh up` | 启动或创建开发环境 | +| `./tool/cicd/run-dev.sh restart` | 重启正在运行的容器 | +| `./tool/cicd/run-dev.sh logs` | 查看并跟踪容器日志 | +| `./tool/cicd/run-dev.sh enter` | 访问容器的 bash shell | +| `./tool/cicd/run-dev.sh down` | 停止并移除容器 | + +## 使用 Docker 实时调试源代码 + +如果你更喜欢运行单个容器进行调试： ```bash -# 先进入项目根目录（该目录包含 `src`），然后执行： +# 首先，cd 到项目根目录（包含 `src`），然后运行： docker run -it \ -v $(pwd):/root/app/ \ -w /root/app/src \ @@ -11,14 +49,14 @@ docker run -it \ -p 8080:80 \ quay.io/labnow/base:latest bash -python -m aloha.script.start app_common.debug +python3 main.py app_common.debug ``` ## 构建 Docker 镜像 ```bash source tool/tool.sh -build_image app_common latest tool/app.Dockerfile +build_image app_common latest tool/app-demo.Dockerfile ``` ## 开发文档 diff --git a/doc/zh/README-get-start.md b/doc/zh/README-get-start.md index c35c18e..17db4e9 100644 --- a/doc/zh/README-get-start.md +++ b/doc/zh/README-get-start.md @@ -8,8 +8,8 @@ pip install aloha[all] ## 第二步：把本仓库当作 boilerplate 使用 -仓库中的 `app/` 目录就是基于 `aloha` 的 boilerplate / 模板项目。 -它已经准备好了可直接使用的项目结构、开发脚本和容器化工具，你可以直接在这个骨架上开始开发，而不需要从零搭建工程。 +本仓库是一个基于 `aloha` 的样板/模板项目。 +它提供了一个即用的应用程序布局、开发脚本和容器化工具，让你无需从头搭建项目骨架，即可开始构建。 ### 这个模板提供了什么 @@ -21,7 +21,7 @@ pip install aloha[all] ### 推荐使用方式 1. 克隆本仓库。 -2. 打开 `app/` 目录，查看模板项目的结构。 +2. 检查启动应用程序的结构。 3. 需要可复现开发环境时，使用 `tool/cicd/` 里的脚本启动开发容器。 4. 在模板结构上放入你自己的业务代码，并逐步扩展。 @@ -34,18 +34,83 @@ pip install aloha[all] ./tool/cicd/run-dev.sh enter ``` -其中 `up` 用于创建或启动开发容器，`enter` 用于进入容器内部的交互式 Shell。 +其中 `up` 命令用于创建或启动开发容器，`enter` 命令用于进入容器内部的交互式 Shell。 ### 项目结构 这个模板围绕几个常见目录组织： -- `app/`：应用代码和入口 -- `src/`：`aloha` 库源码 -- `doc/`：文档源码 -- `notebook/`：用于实验和探索的 Jupyter Notebook -- `tool/`：开发与 CI/CD 相关脚本和 Docker 资源 +- `doc/`：文档源文件。你可以在这里放置你的项目文档。 +- `src/`：应用程序代码和入口。这是你的业务逻辑所在。它包含一个演示应用程序 (`app_common`)，展示了如何使用 `aloha`。 +- `tool/`：开发和 CI/CD 相关的脚本和 Docker 资源。 -你可以参考 GitHub 仓库中的 `app` 目录，在自己的项目中开始使用 `aloha`： +### 如何在你的项目中使用 `aloha` 包 -[:octicons-mark-github-16: 前往模板项目](https://github.com/LabNow-ai/aloha-python/tree/main/app){ .md-button } +`src/` 目录包含一个演示应用程序，展示了如何使用 `aloha` 包。以下是如何在常规 Python 项目开发中导入和使用 `aloha` 的简要概述： + +1. **导入 `aloha` 模块**：为你的应用程序逻辑导入必要的 `aloha` 模块。例如，`aloha.logger` 用于日志记录，`aloha.config` 用于配置，或 `aloha.db` 用于数据库交互。 + +2. **配置你的应用程序**：在 `src/resource/config/main.conf` 中定义你的应用程序配置。 + +3. **实现你的应用程序逻辑**：使用导入的 `aloha` 模块编写你的 Python 代码。 + +4. **运行你的应用程序**：你可以使用提供的 `main.py` 脚本运行应用程序： + +```bash +python3 src/main.py your_module.main +``` + +此命令指示 `src/main.py` 查找并执行你指定模块（例如 `your_module.main`）中的 `main()` 函数。 + +## HOCON 配置 + +`aloha` 使用 HOCON (Human-Optimized Config Object Notation) 格式作为其配置文件。这允许分层、模块化和人类可读的配置。 + +### 配置文件位置 + +默认情况下，`aloha` 会在 `src/resource/config/` 目录中查找配置文件。主配置文件是 `main.conf`。 + +### 模块化配置 + +HOCON 支持包含其他配置文件，这对于模块化你的设置非常有用（例如，分离开发、预发布和生产环境的配置）。 + +**`main.conf` 示例：** + +```hocon +include "deploy-DEV.conf" + +app_name = "MyAlohaApp" + +server { + host = "0.0.0.0" + port = 8080 +} + +database { + type = "postgresql" + connection_string = "${?DB_CONNECTION_STRING}" # 环境变量覆盖 +} +``` + +在此示例中： +- `include "deploy-DEV.conf"` 引入了来自另一个文件的设置，允许进行特定于环境的覆盖。 +- `app_name` 和 `server` 定义了应用程序特定的设置。 +- `database.connection_string = "${?DB_CONNECTION_STRING}"` 演示了如何使用环境变量覆盖配置值，使其适用于不同的部署环境。 + +### 在代码中访问配置 + +你可以通过 `aloha.settings.SETTINGS.config` 在你的 Python 代码中访问配置值： + +```python +from aloha.settings import SETTINGS + +app_name = SETTINGS.config.get("app_name") +server_port = SETTINGS.config.get("server.port") + +print(f"Application Name: {app_name}") +print(f"Server Port: {server_port}") +``` + +这种方法确保了你的应用程序可配置并适应各种部署场景。 + +[:octicons-mark-github-16: 前往模板项目](https://github.com/LabNow-ai/aloha-python/tree/main/src){ .md-button }