doc updates

doc/en/README-develop.md

@@ -1,24 +1,62 @@
 # Development Docs
 
+## Quick Start Development Environment
+
+The project provides a containerized development environment using Docker and Docker Compose. It sets up a complete development workspace with all necessary dependencies pre-installed.
+
+### Launch the Development Environment
+
+Run this command in your terminal:
+
+```bash
+./tool/cicd/run-dev.sh up
+```
+
+This will build the Docker image and start the container. Your local code directories are mounted into the container, enabling live development.
+
+### Enter the Development Container
+
+Once the environment is running, execute:
+
+```bash
+./tool/cicd/run-dev.sh enter
+```
+
+You will be attached to the running container with a bash shell.
+
+### Manage the Environment
+
+The `run-dev.sh` script provides several commands to manage your development environment:
+
+| Command                          | Description                                 |
+| -------------------------------- | ------------------------------------------- |
+| `./tool/cicd/run-dev.sh up`      | Start or create the development environment |
+| `./tool/cicd/run-dev.sh restart` | Restart the running container               |
+| `./tool/cicd/run-dev.sh logs`    | View and follow container logs              |
+| `./tool/cicd/run-dev.sh enter`   | Access the container's bash shell           |
+| `./tool/cicd/run-dev.sh down`    | Stop and remove the container               |
+
 ## Live debug source code with Docker
 
+If you prefer to run a single container for debugging:
+
 ```bash
-# First, cd to project root (which includes `src`), then run:
+# First, cd to project root, then run:
 docker run -it \
   -v $(pwd):/root/app/ \
   -w /root/app/src \
   --name="app-$(whoami)" \
   -p 8080:80 \
   quay.io/labnow/base:latest bash
 
-python -m aloha.script.start app_common.debug
+python3 main.py app_common.debug
 ```
 
 ## Build Docker image
 
 ```bash
 source tool/tool.sh
-build_image app_common latest tool/app.Dockerfile
+build_image app_common latest tool/app-demo.Dockerfile
 ```
 
 ## Develop docs

doc/en/README-get-start.md

@@ -8,7 +8,7 @@ pip install aloha[all]
 
 ## Step 2. Use this repository as a boilerplate
 
-The `app/` folder in this repository is a boilerplate/template project built on top of `aloha`.
+This repository serves as a boilerplate/template project built on top of `aloha`.
 It gives you a ready-to-use application layout, development scripts, and containerized tooling so you can start building instead of assembling the project skeleton yourself.
 
 ### What this template gives you
@@ -21,7 +21,7 @@ It gives you a ready-to-use application layout, development scripts, and contain
 ### Recommended workflow
 
 1. Clone this repository.
-2. Open the `app/` directory to inspect the starter application structure.
+2. Inspect the starter application structure.
 3. Use the scripts under `tool/cicd/` to start the development container when you want a reproducible environment.
 4. Put your own application code in the template structure and grow from there.
 
@@ -40,10 +40,77 @@ The `up` command creates or starts the development container. The `enter` comman
 
 The template is organized around a few common folders:
 
-- `app/`: application code and entry points
-- `src/`: the `aloha` library source code
-- `doc/`: documentation source files
-- `notebook/`: Jupyter notebooks for experimentation
-- `tool/`: scripts and Docker assets for development and CI/CD
+- `doc/`: Documentation source files. You can place your project's documentation here.
+- `src/`: Application code and entry points. This is where your business logic goes. It contains a demo application (`app_common`) showing how to use `aloha`.
+- `tool/`: Scripts and Docker assets for development and CI/CD.
 
-[:octicons-mark-github-16: Go to Template Project](https://github.com/LabNow-ai/aloha-python/tree/main/app){ .md-button }
+### How to use `aloha` in your project
+
+The `src/` directory contains a demo application that demonstrates how to use the `aloha` package. Here is a brief overview of how to import and use `aloha` for regular Python project development:
+
+1. **Import `aloha` modules**: Import the necessary `aloha` modules for your application logic. For example, `aloha.logger` for logging, `aloha.config` for configuration, or `aloha.db` for database interactions.
+
+2. **Configure your application**: Define your application configuration in `src/resource/config/main.conf`.
+
+3. **Implement your application logic**: Write your Python code using the imported `aloha` modules.
+
+4. **Run your application**: You can run your application using the provided `main.py` script:
+
+```bash
+python3 src/main.py your_module.main
+```
+
+This command tells `src/main.py` to find and execute the `main()` function within your specified module (e.g., `your_module.main`).
+
+## Configuration with HOCON
+
+`aloha` uses the HOCON (Human-Optimized Config Object Notation) format for its configuration files. This allows for hierarchical, modular, and human-readable configurations.
+
+### Configuration File Location
+
+By default, `aloha` looks for configuration files in the `src/resource/config/` directory. The primary configuration file is `main.conf`.
+
+### Modular Configuration
+
+HOCON supports including other configuration files, which is useful for modularizing your settings (e.g., separating development, staging, and production configurations).
+
+**Example `main.conf`:**
+
+```hocon
+include "deploy-DEV.conf"
+
+app_name = "MyAlohaApp"
+
+server {
+    host = "0.0.0.0"
+    port = 8080
+}
+
+database {
+    type = "postgresql"
+    connection_string = "${?DB_CONNECTION_STRING}" # Environment variable override
+}
+```
+
+In this example:
+- `include "deploy-DEV.conf"` brings in settings from another file, allowing for environment-specific overrides.
+- `app_name` and `server` define application-specific settings.
+- `database.connection_string = "${?DB_CONNECTION_STRING}"` demonstrates how to use environment variables to override configuration values, making it flexible for different deployment environments.
+
+### Accessing Configuration in Code
+
+You can access configuration values in your Python code via `aloha.settings.SETTINGS.config`:
+
+```python
+from aloha.settings import SETTINGS
+
+app_name = SETTINGS.config.get("app_name")
+server_port = SETTINGS.config.get("server.port")
+
+print(f"Application Name: {app_name}")
+print(f"Server Port: {server_port}")
+```
+
+This approach ensures that your application remains configurable and adaptable to various deployment scenarios.
+
+[:octicons-mark-github-16: Go to Template Project](https://github.com/LabNow-ai/aloha-python/tree/main/src){ .md-button }

doc/skills/aloha_package_usage.md

@@ -0,0 +1,102 @@
+# Aloha Package Usage and API Development Skill
+
+This skill provides guidance on how to effectively use the `aloha` Python package for developing applications within the `aloha-python` boilerplate structure.
+
+## 1. Importing and Using the `aloha` Package
+
+The `aloha` package is designed to be easily integrated into your Python projects. Once installed (e.g., `pip install aloha[all]`), you can import its modules as needed.
+
+### Example: Basic `aloha` Module Usage
+
+To use `aloha` modules, you simply import them and use their functionalities. For example, using `aloha.logger`:
+
+```python
+from aloha.logger import LOG
+
+def my_function():
+    LOG.info("This is an informational message from aloha logger.")
+    return "Function executed successfully"
+```
+
+In this example:
+- `aloha.logger.LOG` provides a pre-configured logger instance.
+
+## 2. Application Configuration with HOCON
+
+`aloha` applications are configured using HOCON (Human-Optimized Config Object Notation) files, typically located in `src/resource/config/`. The `aloha.config.paths` module helps in discovering these configuration files.
+
+### Configuration File Location and Structure
+
+By default, `aloha` looks for configuration files in the `src/resource/config/` directory. The primary configuration file is `main.conf`.
+
+HOCON allows for hierarchical, modular, and human-readable configurations. You can include other configuration files, which is useful for modularizing your settings (e.g., separating development, staging, and production configurations).
+
+**Example `main.conf`:**
+
+```hocon
+include "deploy-DEV.conf"
+
+app_name = "MyAlohaApp"
+
+server {
+    host = "0.0.0.0"
+    port = 8080
+}
+
+database {
+    type = "postgresql"
+    connection_string = "${?DB_CONNECTION_STRING}" # Environment variable override
+}
+```
+
+In this example:
+- `include "deploy-DEV.conf"` brings in settings from another file, allowing for environment-specific overrides.
+- `app_name` and `server` define application-specific settings.
+- `database.connection_string = "${?DB_CONNECTION_STRING}"` demonstrates how to use environment variables to override configuration values, making it flexible for different deployment environments.
+
+### Accessing Configuration in Code
+
+You can access configuration values in your Python code via `aloha.settings.SETTINGS.config`:
+
+```python
+from aloha.settings import SETTINGS
+
+config = SETTINGS.config
+print(f"Application Name: {config.get("app_name")}")
+print(f"Server Port: {config.get("server.port")}")
+```
+
+This approach ensures that your application remains configurable and adaptable to various deployment scenarios.
+
+## 3. Running Your `aloha`-based Application
+
+To run your `aloha`-based application, you can use the `src/main.py` script, which acts as a generic module runner:
+
+### Example: Application Entry Point
+
+Your application's main entry point should be a function (e.g., `main()`) within a module in your `src/` directory. For example, in `src/my_app/main.py`:
+
+```python
+def main():
+    # Your application's startup logic here
+    print("My Aloha application is starting...")
+    # Integrate with FastAPI or other frameworks here
+```
+
+### Running the Application
+
+To execute your application, use the `src/main.py` script:
+
+```bash
+python3 src/main.py my_app.main
+```
+
+This command tells `src/main.py` to find and execute the `main()` function within your specified module (e.g., `my_app.main`).
+
+## 4. Advanced Usage and Best Practices
+
+-   **Configuration Management**: Leverage `aloha.config.paths` and HOCON files for environment-specific configurations (e.g., `deploy-DEV.conf`). This allows for modular and flexible management of settings across different environments.
+-   **HOCON Configuration Details**: For a comprehensive understanding of HOCON configuration, refer to the "Configuration with HOCON" section in the `README-get-start.md` documentation.
+-   **Database Integration**: Utilize `aloha.db` modules (e.g., `aloha.db.postgres`) for seamless database interactions, as demonstrated in `src/app_common/api/api_common_query_postgres.py`.
+-   **Logging**: Use `aloha.logger.LOG` for consistent and configurable logging across your application.
+-   **Testing**: Employ `aloha.testing` utilities for unit and integration tests.

doc/skills/dev_env_cicd.md

@@ -0,0 +1,72 @@
+# Development Environment and CI/CD Management Skill
+
+This skill outlines the conventions and tools used within the `aloha-python` boilerplate for managing the development environment and CI/CD processes. It focuses on leveraging Docker and Docker Compose for a consistent and reproducible development workflow.
+
+## 1. Containerized Development Environment
+
+The `aloha-python` boilerplate provides a fully containerized development environment using Docker and Docker Compose. This ensures that all developers work with the same dependencies and configurations, minimizing "it works on my machine" issues.
+
+### Quick Start with `run-dev.sh`
+
+The primary script for managing the development environment is `./tool/cicd/run-dev.sh`. This script simplifies common Docker operations.
+
+**To launch the development environment:**
+
+```bash
+./tool/cicd/run-dev.sh up
+```
+
+This command performs the following actions:
+-   **Port Availability Check**: Verifies that required ports (dynamically assigned based on user ID to avoid conflicts) are free.
+-   **Docker Image Build**: If the image doesn't exist, it builds it using `tool/dev-demo.Dockerfile` and `tool/cicd/docker-compose.app-demo.DEV.yml`. The build process includes installing Node.js (pnpm), Python with JupyterLab, project dependencies, and PostgreSQL client tools.
+-   **Container Start**: Starts the Docker container with volume mounts (e.g., `doc/`, `notebook/`, `src/` are mounted to `/root/app/doc`, `/root/app/notebook`, `/root/app/src` respectively) and port forwarding.
+
+**To enter the development container:**
+
+```bash
+./tool/cicd/run-dev.sh enter
+```
+
+This command provides an interactive bash shell inside the running container, with the working directory set to `/root/app`.
+
+### `run-dev.sh` Commands
+
+The `run-dev.sh` script supports the following commands for environment management:
+
+| Command                          | Description                                 |
+| :------------------------------- | :------------------------------------------ |
+| `./tool/cicd/run-dev.sh up`      | Starts or creates the development environment. |
+| `./tool/cicd/run-dev.sh restart` | Restarts the running container.             |
+| `./tool/cicd/run-dev.sh logs`    | Views and follows container logs.           |
+| `./tool/cicd/run-dev.sh enter`   | Accesses the container's bash shell.        |
+| `./tool/cicd/run-dev.sh down`    | Stops and removes the container.            |
+
+## 2. Dockerfile and Docker Compose Configuration
+
+-   **`tool/dev-demo.Dockerfile`**: This Dockerfile defines the base image for the development container. It includes installations for `pnpm`, `jupyterlab`, Python dependencies, and database clients.
+-   **`tool/cicd/docker-compose.app-demo.DEV.yml`**: This Docker Compose file orchestrates the development environment. It specifies how the Docker image is built, environment variables (including `PYTHONPATH` for easy imports from `pkg/` and `src/`), exposed ports, and volume mounts to enable live code changes.
+
+## 3. Building Docker Images for Deployment
+
+The `tool/tool.sh` script provides utilities for building and managing Docker images for deployment.
+
+**To build a Docker image for your application:**
+
+```bash
+source tool/tool.sh
+build_image app_common latest tool/app-demo.Dockerfile
+```
+
+This command uses `tool/app-demo.Dockerfile` to build a production-ready Docker image for your application.
+
+## 4. Developing Documentation Locally
+
+To develop and preview documentation locally, the project uses MkDocs.
+
+**To serve the documentation locally:**
+
+```bash
+mkdocs serve -f mkdocs.yml -a 0.0.0.0:3000
+```
+
+This command starts a local web server, allowing you to view changes to your Markdown documentation in real-time. The main configuration files are `doc/mkdocs.yml` (for English) and `doc/mkdocs.zh.yml` (for Chinese).

doc/skills/project_scaffolding.md

@@ -0,0 +1,34 @@
+# Project Scaffolding and Conventions Skill
+
+This skill provides guidelines for understanding and utilizing the `aloha-python` repository as a boilerplate for new Python projects. It outlines the project structure, the purpose of key directories, and how to leverage the `aloha` package within this framework.
+
+## 1. Project Structure Overview
+
+The `aloha-python` repository is organized into several top-level directories, each serving a specific purpose:
+
+- **`doc/`**: This directory contains all project documentation, including guides, API references, and these AI Agent Skills. When creating new documentation, it should be placed here, typically organized by language (e.g., `doc/en`, `doc/zh`).
+
+- **`pkg/`**: This directory stores the source code for the `aloha` Python package that is intended for publication to PyPI. It is the correct place to modify when the task is to work on this library itself. When using this repository as a boilerplate for a new application project, developers or agents should not include this directory unless they explicitly intend to create and publish a new package to PyPI.
+
+- **`src/`**: This directory is designed for the application-specific code that consumes the `aloha` package. It serves as a boilerplate example (`app_common`) for how to structure a project using `aloha`. New projects based on this boilerplate should place their primary application logic and modules here.
+
+- **`notebook/`**: This directory is for Jupyter notebooks, which can be used for experimentation, data analysis, or interactive development related to the project.
+
+- **`tool/`**: This directory contains development and CI/CD related scripts and Docker assets. This includes scripts for setting up the development environment, building Docker images, and managing the project lifecycle.
+
+## 2. Using the Repository as a Boilerplate
+
+To initiate a new project using `aloha-python` as a boilerplate, follow these steps:
+
+1.  **Clone the Repository**: Begin by cloning the `aloha-python` repository to your local machine.
+2.  **Inspect `src/`**: Review the `src/` directory to understand the example application structure (`app_common`) and how it integrates with the `aloha` package.
+3.  **Develop Your Application**: Place your application-specific code within the `src/` directory, following the established patterns for modularity and `aloha` integration.
+4.  **Utilize `tool/cicd/`**: Leverage the scripts provided in `tool/cicd/` for managing your development environment, as detailed in the "Development Environment and CI/CD Management Skill".
+
+## 3. Key Conventions for Boilerplate Projects
+
+-   **`src/` for Application Logic**: All primary application code, including API handlers, business logic, and utility modules, should reside within `src/`. The `src/main.py` script acts as a generic entry point for running Python modules within the `src/` directory. Your application's main function should be callable via `python3 src/main.py your_module.main`.
+-   **`pkg/` is not part of a new boilerplate app**: If the goal is to build a new application project from this repository, do not carry over `pkg/` unless the user specifically wants to create and publish a separate package. Application code should live in `src/` instead.
+-   **`resource/config/` for Configuration**: Application configuration files (e.g., `main.conf`, `deploy-DEV.conf`) should be placed under `src/resource/config/`. The `aloha` package's `aloha.config.paths` module handles the discovery and loading of these configuration files. For detailed information on HOCON configuration, refer to the "Configuration with HOCON" section in the `aloha_package_usage.md` skill.
+
+By adhering to these conventions, AI agents can effectively understand, navigate, and contribute to projects built upon the `aloha-python` framework.