From a28e4d4864618b6bb8a57a183ca73f8c03b374fa Mon Sep 17 00:00:00 2001 From: usualalteration Date: Tue, 16 Jun 2026 17:22:49 +0200 Subject: [PATCH 01/10] First tutorial-ai modified --- content/en/docs/tutorial-ai/Lesson0/_index.md | 116 +++++++++--------- 1 file changed, 55 insertions(+), 61 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson0/_index.md b/content/en/docs/tutorial-ai/Lesson0/_index.md index 24658312..6919a93b 100644 --- a/content/en/docs/tutorial-ai/Lesson0/_index.md +++ b/content/en/docs/tutorial-ai/Lesson0/_index.md @@ -5,10 +5,9 @@ draft: true --- # TO BE REVIEWED! +###### 1st review 16/06/2026 -# πŸ‘‹ Welcome to the Course: Application Development with LLM Open and Apache Open Serverless - -A mileage title worthy of WertmΓΌller β€” but for friends, it's simply the **Mastro GPT Course**. +# πŸ‘‹ Welcome to the Course: Application Development with Apache Open Serverless --- @@ -22,7 +21,7 @@ To start working, launch the development environment. 3. Select **"Create codespace on main"**. 4. Wait for it to start (takes some time the first time). -> This is the environment used for the course. It's convenient and preconfigured for both training and development. +This is the environment used for the course. It's convenient and preconfigured for both training and development. --- @@ -45,12 +44,7 @@ Once your Codespace is running, take note of these key icons in the sidebar: 4. You should see the message: You have successfully logged in. You can now use Open Serverless. - -yaml -Copy -Edit - -This concludes the initial setup. This is **Lesson Zero**, a pre-lesson to verify that everything works. +This concludes the initial setup. --- @@ -81,109 +75,108 @@ If you don’t see the **Open in browser** button: ## πŸ€– Step 6: Meet "Pinocchio" β€” The User Interface -The web frontend is named **Pinocchio**, a reference to Mastro Geppetto. - +The web frontend is named **Pinocchio**, a reference to Carlo Collodi's book. ### Default login credentials: - **Username**: Pinocchio - **Password**: Geppetto (or vice versa) -> You’ll be prompted to change it. Here’s how. +You’ll be prompted to change it. --- ## πŸ” Step 7: Change the Password (via Terminal) -1. Open the terminal: -Terminal β†’ New Terminal +1. Open the terminal: Terminal β†’ New Terminal 2. Run the following command: -bash -obs ai user update Pinocchio -Enter your new password. +```ops ai user update Pinocchio``` -Redeploy the login service: +3. Enter your new password. + +4. Redeploy the login service: + +```ops ide deploy mastrogpt-login``` -bash -Copy -Edit -obs deploy mastrogpt-login This shows how command-line tools can be used instead of the GUI for advanced operations. -πŸ§‘β€πŸ’» Step 8: Tour the Interface Features -Pinocchio is a multi-chat UI developed in Python. You won't need to change the UI β€” you'll build backend logic for chat apps. +## πŸ§‘β€πŸ’» Step 8: Tour the Interface Features + +> Pinocchio is a multi-chat UI developed in Python. You won't need to change the UI β€” you'll build backend logic for chat apps. Available chats: + hello: Responds with a greeting. demo: Demonstrates Pinocchio’s interface features: -Code rendering +- Code rendering -HTML view +- HTML view -Chessboard display +- Chessboard display -Forms and form submissions +- Forms and form submissions -Document uploads +- Document uploads -Custom side views +- Custom side views Everything is extendable and customizable. -πŸ§ͺ Step 9: View the Slides +## πŸ§ͺ Step 9: View the Slides + To view this lesson's materials: -Click the πŸ“„ Docs icon. +1. Click the πŸ“„ Docs icon. -Navigate to the lessons/ folder. +2. Navigate to the lessons/folder. -Open the lesson markdown file. +3. Open the lesson markdown file. -Use the Preview tab to view the slide. +4. Use the Preview tab to view the slide. -Use Source to copy exercises or commands. +5. Use Source to copy exercises or commands. -πŸ’‘ Step 10: Tips for Codespaces -To avoid wasting free hours: +## πŸ’‘ Step 10: Tips for Codespaces -Go to GitHub β†’ Settings β†’ Codespaces. +To avoid wasting the free hours of Github Codespaces: -Set Auto-off timeout to 5–10 minutes. +1. Go to GitHub β†’ Settings β†’ Codespaces. -Optionally switch from VS Code web to VS Code desktop. +2. Set Auto-off timeout to 5–10 minutes. -Alternatively, install and run everything locally using Docker. +3. Optionally switch from VS Code web to VS Code desktop. -We can organize a dedicated lesson on local installation if needed. +Alternatively, install and run everything locally using Docker. -☁️ Step 11: Open Serverless Services +## ☁️ Step 11: Open Serverless Services Your environment includes: -Redis +- Redis -MinIO +- MinIO -PostgreSQL (not required for this course) +- PostgreSQL (not required for this course) You can deploy apps on: -AWS +- AWS + +- Google Cloud -Google Cloud +- Azure -Azure +- Akamai -Akamai +- Hetzner -Hetzner +- Ubuntu -Ubuntu +- OpenShift -OpenShift +## πŸ†˜ Step 12: Get Support -πŸ†˜ Step 12: Get Support You can request your free account or support via: 🌐 Website: https://mastrogpt.com @@ -196,15 +189,16 @@ You can request your free account or support via: πŸ—£οΈ Reddit: Ask questions and join the discussion -▢️ Step 13: Start Lesson 1 +## ▢️ Step 13: Start Lesson 1 + Now that everything is set up: -Go back to the Cloud icon. +1. Go back to the Cloud icon. -Click Lessons. +2. Click Lessons. -Select Lesson 1. +3. Select Lesson 1. -All lesson files will be downloaded automatically. +4. All lesson files will be downloaded automatically. You're now ready to start building! \ No newline at end of file From 2e6865bd592e8e108bf4da592cb66b12c86b02ea Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 16:29:31 +0200 Subject: [PATCH 02/10] fixed #79 --- content/en/docs/tutorial-ai/Lesson1/_index.md | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson1/_index.md b/content/en/docs/tutorial-ai/Lesson1/_index.md index 655bdff9..b7aebc92 100644 --- a/content/en/docs/tutorial-ai/Lesson1/_index.md +++ b/content/en/docs/tutorial-ai/Lesson1/_index.md @@ -5,11 +5,13 @@ draft: true --- # TO BE REVIEWED! +###### 1st review 17/06/2026 +> βœ… _Lesson 0_ was the environment setup. _Lesson 1_ is the **first real lesson** of the course. # πŸ“˜ Lesson 1 – Integrated Services & First Exercise -Welcome! After launching and configuring your environment, ensure you're **logged in** to begin. +After launching and configuring your environment, ensure you're **logged in** to begin. --- @@ -19,8 +21,6 @@ Welcome! After launching and configuring your environment, ensure you're **logge 2. This will automatically download all lesson files. 3. You can preview the new lesson by selecting the markdown file under `lessons/`. -> βœ… _Lesson 0_ was the environment setup. _Lesson 1_ is the **first real lesson** of the course. - --- ## 🧰 Included "Hello" Examples @@ -56,13 +56,13 @@ The `hello` package includes small examples demonstrating system services: ```bash # Deploy the fixed code -obs deploy +ops ide deploy # Or just redeploy the specific action -obs ide deploy hello-llm +ops ide deploy hello-llm ``` -> Enable **Dev Mode** (Devil) for automatic deploy on save. +> Enable **Dev Mode** for automatic deploy on save. --- @@ -114,18 +114,18 @@ After deployment, you’ll get a **custom URL**. Login with your user credential ## πŸ”§ Command-Line Tools (OBS CLI) -Use `obs` or `oops` for CLI actions. Common subcommands: +Use `ops` for CLI actions. Common subcommands: ```bash -obs ai lesson # download lessons -obs ai user # manage users -obs ai chat # chat with an LLM from CLI -obs ide deploy # deploy a single action -obs action delete # delete an action -obs clean # cleanup temp files +ops ai lesson # download lessons +ops ai user # manage users +ops ai chat # chat with an LLM from CLI +ops ide deploy # deploy a single action +ops action delete # delete an action +ops clean # cleanup temp files ``` -> `obs ai chat` is a terminal version of the web chat. +> `ops ai chat` is a terminal version of the web chat. --- @@ -155,7 +155,7 @@ packages/ 4. Deploy: ```bash -obs deploy +ops deploy ``` 5. Add to UI: @@ -167,7 +167,7 @@ obs deploy 6. Redeploy the index: ```bash -obs ide deploy mastrogpt-index +ops ide deploy mastrogpt-index ``` 7. Try it in the web UI: @@ -213,4 +213,4 @@ Next lessons will: - Cover vision models and embeddings - Show full-stack app development using Open Serverless -Happy hacking! πŸ§ͺπŸ€– +Happy coding! πŸ§ͺπŸ€– From ef94feea24bb37951f539c1b4ac4706c6f593784 Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 17:36:57 +0200 Subject: [PATCH 03/10] fixed #78 fixed #79 tutorial-ai 0 and 1 --- content/en/docs/tutorial-ai/Lesson0/_index.md | 59 +++++++++---------- content/en/docs/tutorial-ai/Lesson1/_index.md | 42 +++++++------ 2 files changed, 48 insertions(+), 53 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson0/_index.md b/content/en/docs/tutorial-ai/Lesson0/_index.md index 6919a93b..9d9afb93 100644 --- a/content/en/docs/tutorial-ai/Lesson0/_index.md +++ b/content/en/docs/tutorial-ai/Lesson0/_index.md @@ -4,18 +4,15 @@ weight: 10 draft: true --- -# TO BE REVIEWED! -###### 1st review 16/06/2026 - -# πŸ‘‹ Welcome to the Course: Application Development with Apache Open Serverless +# Welcome to the Course: Application Development with Apache Open Serverless --- -## πŸš€ Step 1: Start the Environment +## Step 1: Start the Environment To start working, launch the development environment. -### 🟒 Recommended: GitHub Codespace +### Recommended: GitHub Codespace 1. Go to **GitHub β†’ Mastro GPT**: https://github.com/mastrogpt 2. Click the **"Code"** button. 3. Select **"Create codespace on main"**. @@ -25,18 +22,18 @@ This is the environment used for the course. It's convenient and preconfigured f --- -## βš™οΈ Step 2: Explore the Interface Icons +## Step 2: Explore the Interface Icons Once your Codespace is running, take note of these key icons in the sidebar: -- ☁️ **Cloud icon**: Opens the Open Serverless extension. -- πŸ§ͺ **Test tube icon**: Opens the list of tests. -- πŸ“„ **Docs icon**: Opens course documents and slides. -- πŸ” **Search icon**: Lets you search in your code and documents. +- **Cloud icon**: Opens the Open Serverless extension. +- **Test tube icon**: Opens the list of tests. +- **Docs icon**: Opens course documents and slides. +- **Search icon**: Lets you search in your code and documents. --- -## πŸ” Step 3: Login to Open Serverless +## Step 3: Login to Open Serverless 1. Click the **Cloud icon**. 2. Press the **Login** button. @@ -48,7 +45,7 @@ This concludes the initial setup. --- -## πŸ§ͺ Step 4: Deploy and Run Tests +## Step 4: Deploy and Run Tests To ensure your setup is working: @@ -59,7 +56,7 @@ To ensure your setup is working: --- -## 🧰 Step 5: Use Development Mode +## Step 5: Use Development Mode Switch to development mode to run the web interface: @@ -68,12 +65,12 @@ Switch to development mode to run the web interface: If you don’t see the **Open in browser** button: -- Look for the πŸ“‘ **Antenna icon** at the bottom bar. -- Click the 🌐 **Globe icon** to open the UI in a browser. +- Look for the **Antenna icon** at the bottom bar. +- Click the **Globe icon** to open the UI in a browser. --- -## πŸ€– Step 6: Meet "Pinocchio" β€” The User Interface +## Step 6: Meet "Pinocchio" β€” The User Interface The web frontend is named **Pinocchio**, a reference to Carlo Collodi's book. ### Default login credentials: @@ -84,7 +81,7 @@ You’ll be prompted to change it. --- -## πŸ” Step 7: Change the Password (via Terminal) +## Step 7: Change the Password (via Terminal) 1. Open the terminal: Terminal β†’ New Terminal @@ -100,7 +97,7 @@ You’ll be prompted to change it. This shows how command-line tools can be used instead of the GUI for advanced operations. -## πŸ§‘β€πŸ’» Step 8: Tour the Interface Features +## Step 8: Tour the Interface Features > Pinocchio is a multi-chat UI developed in Python. You won't need to change the UI β€” you'll build backend logic for chat apps. @@ -124,11 +121,11 @@ demo: Demonstrates Pinocchio’s interface features: Everything is extendable and customizable. -## πŸ§ͺ Step 9: View the Slides +## Step 9: View the Slides To view this lesson's materials: -1. Click the πŸ“„ Docs icon. +1. Click the Docs icon. 2. Navigate to the lessons/folder. @@ -138,7 +135,7 @@ To view this lesson's materials: 5. Use Source to copy exercises or commands. -## πŸ’‘ Step 10: Tips for Codespaces +## Step 10: Tips for Codespaces To avoid wasting the free hours of Github Codespaces: @@ -150,7 +147,7 @@ To avoid wasting the free hours of Github Codespaces: Alternatively, install and run everything locally using Docker. -## ☁️ Step 11: Open Serverless Services +## Step 11: Open Serverless Services Your environment includes: - Redis @@ -175,21 +172,21 @@ You can deploy apps on: - OpenShift -## πŸ†˜ Step 12: Get Support +## Step 12: Get Support You can request your free account or support via: -🌐 Website: https://mastrogpt.com +- Website: https://mastrogpt.com -πŸ’¬ Chatbot on the website +- Chatbot on the website -πŸ’Ό LinkedIn: Send us a message +- LinkedIn: Send us a message -πŸ’» Discord: Primary support (use the Italian channel if preferred) +- Discord: Primary support (use the Italian channel if preferred) -πŸ—£οΈ Reddit: Ask questions and join the discussion +- Reddit: Ask questions and join the discussion -## ▢️ Step 13: Start Lesson 1 +## Step 13: Start Lesson 1 Now that everything is set up: @@ -201,4 +198,4 @@ Now that everything is set up: 4. All lesson files will be downloaded automatically. -You're now ready to start building! \ No newline at end of file +You're now ready to start building! diff --git a/content/en/docs/tutorial-ai/Lesson1/_index.md b/content/en/docs/tutorial-ai/Lesson1/_index.md index b7aebc92..aa7ecdac 100644 --- a/content/en/docs/tutorial-ai/Lesson1/_index.md +++ b/content/en/docs/tutorial-ai/Lesson1/_index.md @@ -4,18 +4,16 @@ weight: 10 draft: true --- -# TO BE REVIEWED! -###### 1st review 17/06/2026 -> βœ… _Lesson 0_ was the environment setup. _Lesson 1_ is the **first real lesson** of the course. +> _Lesson 0_ was the environment setup. _Lesson 1_ is the **first real lesson** of the course. -# πŸ“˜ Lesson 1 – Integrated Services & First Exercise +# Lesson 1 – Integrated Services & First Exercise After launching and configuring your environment, ensure you're **logged in** to begin. --- -## πŸ“₯ Downloading the Lesson +## Downloading the Lesson 1. In the extension, select **Lesson 1** from the sidebar. 2. This will automatically download all lesson files. @@ -23,7 +21,7 @@ After launching and configuring your environment, ensure you're **logged in** to --- -## 🧰 Included "Hello" Examples +## Included "Hello" Examples The `hello` package includes small examples demonstrating system services: @@ -40,17 +38,17 @@ The `hello` package includes small examples demonstrating system services: --- -## βœ… Running and Fixing Tests +## Running and Fixing Tests -1. Run the tests (πŸ§ͺ Tests tab). +1. Run the tests (Tests tab). 2. Two tests **intentionally fail**. 3. Locate the `TODO` in the broken test. 4. Example fix: change `"Hi"` to `"Hello"` and save. ### Why Tests Still Fail After Fix? -- βœ… **Unit test** will pass immediately. -- ❌ **Integration test** will still fail β€” it requires deployment! +- **Unit test** will pass immediately. +- **Integration test** will still fail β€” it requires deployment! ### Fix Integration Tests @@ -66,13 +64,13 @@ ops ide deploy hello-llm --- -## 🌐 Using the Deployed Examples +## Using the Deployed Examples After deployment, you’ll get a **custom URL**. Login with your user credentials. ### Example Services Overview -#### 1. πŸ€– Alma (LLM) +#### 1. Alma (LLM) - Powered by a 3.18B parameter model - Works in Italian and English @@ -80,12 +78,12 @@ After deployment, you’ll get a **custom URL**. Login with your user credential - Ask: `What is the capital of Italy?` - Response: `Rome` -#### 2. πŸ”„ Streamer +#### 2. Streamer - Demonstrates streaming via slow ASCII output - Example: `Hi, how are you?` β†’ streamed char-by-char -#### 3. ⚑ Redis +#### 3. Redis - Supports CLI-like interaction - Examples: @@ -94,13 +92,13 @@ After deployment, you’ll get a **custom URL**. Login with your user credential keys * ``` -#### 4. πŸ—‚οΈ Object Storage (MinIO) +#### 4. Object Storage (MinIO) - S3-compatible local storage - Commands: list, insert (`+ key = value`), search, delete - Supports file upload via UI -#### 5. πŸ“š Vector DB (Milvus) +#### 5. Vector DB (Milvus) - Supports semantic search - Example: @@ -112,7 +110,7 @@ After deployment, you’ll get a **custom URL**. Login with your user credential --- -## πŸ”§ Command-Line Tools (OBS CLI) +## Command-Line Tools (OBS CLI) Use `ops` for CLI actions. Common subcommands: @@ -129,7 +127,7 @@ ops clean # cleanup temp files --- -## ✨ Creating a New Service: Reverse Text +## Creating a New Service: Reverse Text ### Step-by-step: "Reverse Text" Chat App @@ -175,7 +173,7 @@ ops ide deploy mastrogpt-index --- -## 🧠 Edit Logic in Code +## Edit Logic in Code File: `packages/messiah/reverse/main.py` @@ -193,7 +191,7 @@ Use **Dev Mode** to auto-deploy as you edit. --- -## βœ… Summary +## Summary You now: @@ -205,7 +203,7 @@ You now: --- -## πŸ§‘β€πŸ« Up Next +## Up Next Next lessons will: @@ -213,4 +211,4 @@ Next lessons will: - Cover vision models and embeddings - Show full-stack app development using Open Serverless -Happy coding! πŸ§ͺπŸ€– +Happy coding! From 14076a64e4a643ee9ba703d31189a7dda7954002 Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 18:58:36 +0200 Subject: [PATCH 04/10] fixed #80 lesson 2 --- content/en/docs/tutorial-ai/Lesson2/_index.md | 464 ++++++++++++++++++ 1 file changed, 464 insertions(+) diff --git a/content/en/docs/tutorial-ai/Lesson2/_index.md b/content/en/docs/tutorial-ai/Lesson2/_index.md index 434031e5..df08b596 100644 --- a/content/en/docs/tutorial-ai/Lesson2/_index.md +++ b/content/en/docs/tutorial-ai/Lesson2/_index.md @@ -6,3 +6,467 @@ draft: true # TO BE REVIEWED! +> _Lesson 1_ introduced the first services and a simple chat action. _Lesson 2_ shows how to call an LLM directly and stream its answer through Open Serverless. + +# Lesson 2 – LLM Access, Secrets & Streaming + +In this lesson you will build an LLM chat that streams its output while the model is generating it. + +You will learn: + +- How to access the Ollama LLM endpoint +- How secrets are passed to tests and deployed actions +- How streaming works in Python and in Open Serverless +- How to test streaming locally with a mock +- How to complete a streamed LLM chat exercise + +--- + +## Preparing the Environment + +Start from the Mastro GPT starter repository as in the previous lessons. + +1. Open the Mastro GPT GitHub organization. +2. Fork the starter repository into your own account. +3. Create a Codespace from your fork. +4. Log in from the Open Serverless extension using your course credentials. +5. Select **Lesson 2** from the lessons panel. + +Each lesson is independent. You do not need the files from Lesson 1 to complete Lesson 2. + +> You can also work locally, but Codespaces keeps the course environment preconfigured. + +--- + +## Optional Editor Shortcut + +During the lesson it is useful to send selected code from the editor directly to the terminal. + +In VS Code: + +1. Open **Settings**. +2. Select **Keyboard Shortcuts**. +3. Search for **Run Selected Text in Active Terminal**. +4. Assign a shortcut, for example `Ctrl+Enter`. + +This makes it easier to copy small code snippets into the interactive CLI while experimenting. + +--- + +## Accessing the LLM + +The LLM endpoint is protected with the same credentials used by Open Serverless. + +The two values you need are: + +| Variable | Purpose | +|---------------|------------------------------------| +| `OLLAMA_HOST` | Hostname of the Ollama server | +| `AUTH` | Credentials used to access Ollama | + +Open the interactive Python CLI: + +```bash +ops ai cli +``` + +Read the values from the environment: + +```python +import os + +args = {} +host = args.get("OLLAMA_HOST", os.getenv("OLLAMA_HOST")) +auth = args.get("AUTH", os.getenv("AUTH")) +base = f"https://{auth}@{host}" +``` + +The pattern is important: + +```python +value = args.get("NAME", os.getenv("NAME")) +``` + +Always read secrets from `args` first, then fall back to the environment. This keeps the same code usable in tests, local CLI experiments, and deployed actions. + +You can verify that Ollama is reachable: + +```python +!curl {base} +``` + +You should receive a response showing that Ollama is running. + +--- + +## Calling Ollama Without Streaming + +Start with a normal request that returns the full answer only when generation is complete. + +```python +import requests + +model = "llama3.1:8b" +inp = "Who are you?" + +msg = { + "model": model, + "prompt": inp, + "stream": False, +} + +url = f"{base}/api/generate" +res = requests.post(url, json=msg).json() +print(res["response"]) +``` + +This is the basic Ollama API flow: + +1. Choose a model. +2. Build a JSON request. +3. POST it to `/api/generate`. +4. Read the `response` field. + +There are Python libraries for Ollama, but using the HTTP API directly is useful because it shows exactly what the service expects. + +--- + +## Creating an LLM Action + +The same logic can be placed inside an Open Serverless action. + +A typical action: + +```python +import os +import requests + +def main(args): + inp = args.get("input", "") + if inp == "": + return {"body": {"output": "Please provide an input"}} + + host = args.get("OLLAMA_HOST", os.getenv("OLLAMA_HOST")) + auth = args.get("AUTH", os.getenv("AUTH")) + base = f"https://{auth}@{host}" + + msg = { + "model": "llama3.1:8b", + "prompt": inp, + "stream": False, + } + + res = requests.post(f"{base}/api/generate", json=msg).json() + return {"body": {"output": res["response"]}} +``` + +You can create or update actions directly from the CLI, but for normal development use: + +```bash +ops ide deploy +``` + +or deploy a single action: + +```bash +ops ide deploy +``` + +--- + +## How Secrets Are Propagated + +Secrets can come from several places: + +| Source | Used for | +|-----------------|-----------------------------------------------| +| Login config | Values loaded when you log in | +| `.env` | Shared project variables | +| `packages/.env` | Variables used for deployed packages/actions | +| test env files | Variables used while running tests | + +The shell does not necessarily see every secret. The `ops` CLI can read the Open Serverless configuration. + +To inspect the loaded configuration: + +```bash +ops config dump +``` + +To search for a specific value: + +```bash +ops config dump | grep OLLAMA +``` + +Action annotations are used to pass configuration values during deployment. They are written as comments at the top of the action file. + +Example: + +```python +#--param OLLAMA_HOST $OLLAMA_HOST +#--param AUTH $AUTH +``` + +When you run: + +```bash +ops ide deploy +``` + +the deploy command reads these annotations and passes the values to the action. + +### What `ops ide deploy` Does + +`ops ide deploy` builds on top of lower-level Open Serverless action and package commands. + +It can: + +- Create packages and actions +- Zip multi-file actions +- Resolve dependencies from files such as `requirements.txt`, `package.json`, or `composer.json` +- Extract annotations from action files +- Propagate secrets into deployed actions +- Work with development mode for incremental deploys + +For normal development, prefer `ops ide deploy` over manually creating actions. + +--- + +## Streaming From Ollama + +Streaming lets the user see the response while the model is still generating it. + +To request streaming from Ollama, set `stream` to `True`: + +```python +import requests + +msg = { + "model": "llama3.1:8b", + "prompt": "Who are you?", + "stream": True, +} + +res = requests.post(f"{base}/api/generate", json=msg, stream=True) +``` + +In Python, the response stream is an iterator: + +```python +for item in res.iter_lines(): + print(item) +``` + +Ollama returns a sequence of JSON objects. Each object contains fields such as: + +| Field | Meaning | +|------------|--------------------------------------| +| `model` | Model that generated the response | +| `response` | Current generated text fragment | +| `done` | Whether generation has finished | + +Only the `response` field is normally sent to the user interface. + +--- + +## Streaming in Open Serverless + +Open Serverless actions are asynchronous, so they do not keep a normal direct connection to the browser. + +For streaming, Open Serverless uses a **streamer** component: + +1. The UI asks for a streamed response. +2. The streamer invokes the action. +3. The action receives `stream_host` and `stream_port`. +4. The action opens a socket to that host and port. +5. The action sends partial output to the socket as it is produced. + +The action should also return: + +```python +{"streaming": True} +``` + +This tells Pinocchio to call the streamed endpoint instead of waiting for a single final response. + +--- + +## Countdown Streaming Example + +Before streaming an LLM, the lesson uses a simple countdown iterator. + +```python +import time + +def count_to_zero(n): + for i in range(n - 1, -1, -1): + time.sleep(1) + yield str(i) +``` + +This is useful because it behaves like a stream without requiring an LLM. + +A streaming helper receives an iterator and sends each item to the stream socket. It also collects the final result so the action can return the full output at the end. + +The same helper can be tested locally with a stream mock. + +--- + +## Testing Streaming With a Mock + +Streaming is hard to test directly because it depends on a socket opened by the runtime. + +The lesson provides a **stream mock** that simulates the streamer locally. + +The test flow is: + +1. Create mock stream arguments. +2. Start the stream mock. +3. Run the streaming function with those arguments. +4. Stop the mock. +5. Assert that the expected streamed output was received. + +The mock waits only a few seconds before timing out, so start the function immediately after starting the mock. + +This lets you test streamed code before deploying it. + +--- + +## Exercise 1: Add LLM Secrets + +Find: + +```text +TODO E2.1 +``` + +Complete the action so it receives the Ollama values. + +Add the parameters: + +```python +#--param OLLAMA_HOST $OLLAMA_HOST +#--param AUTH $AUTH +``` + +Then read them in the code: + +```python +host = args.get("OLLAMA_HOST", os.getenv("OLLAMA_HOST")) +auth = args.get("AUTH", os.getenv("AUTH")) +base = f"https://{auth}@{host}" +``` + +After this exercise, the URL test should pass. The streaming test will still fail until Exercise 2 is completed. + +--- + +## Exercise 2: Decode Ollama Streaming Output + +Find: + +```text +TODO E2.2 +``` + +The existing streaming function is written for the countdown example. Ollama returns JSON objects instead of plain text, so each streamed item must be decoded. + +Use `json.loads` to parse each item: + +```python +import json + +dec = json.loads(item.decode("utf-8")) +out = dec.get("response", "") +``` + +Then send only `out` to the stream and append it to the final result. + +After this change, the streaming test should pass. + +--- + +## Exercise 3: Add a Model Switch + +Find: + +```text +TODO E2.3 +``` + +Add a simple command that lets the user switch models from the chat input. + +Example behavior: + +| Input | Model selected | Prompt sent to the model | +|------------|----------------|--------------------------| +| `llama` | `llama3.1:8b` | `Who are you?` | +| `deepseek` | DeepSeek model | `Who are you?` | + +For example: + +```python +if inp == "llama": + model = "llama3.1:8b" + inp = "Who are you?" +elif inp == "deepseek": + model = "deepseek-r1:7b" + inp = "Who are you?" +``` + +DeepSeek may emit hidden thinking sections. If they appear as unreadable tags, convert them to a readable Markdown form, for example by replacing angle brackets with square brackets. + +--- + +## Deploy and Try the Chat + +Deploy the lesson actions: + +```bash +ops ide deploy +``` + +Open Pinocchio and select the streamed LLM chat. + +Try prompts such as: + +```text +Who are you? +``` + +```text +List the capitals of the United States +``` + +Then test the model switch: + +```text +deepseek +``` + +```text +llama +``` + +The first answer from a newly selected model can take longer because the model may need to load. + +--- + +## Summary + +You now: + +- Accessed Ollama from the CLI +- Built a non-streaming LLM request +- Learned where secrets come from and how actions receive them +- Used action annotations to propagate secret parameters +- Learned how Python iterators represent streams +- Tested streaming locally with a stream mock +- Completed a streamed LLM chat with model switching + +--- + +## Up Next + +Next lessons will build on this foundation by using LLMs in richer applications, including embeddings, vision models, and full-stack Open Serverless workflows. From 686865f7f4828a80686cf20e1d62311a35008148 Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:04:55 +0200 Subject: [PATCH 05/10] fixed #81 lesson 3 --- content/en/docs/tutorial-ai/Lesson2/_index.md | 6 +- content/en/docs/tutorial-ai/Lesson3/_index.md | 449 +++++++++++++++++- 2 files changed, 450 insertions(+), 5 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson2/_index.md b/content/en/docs/tutorial-ai/Lesson2/_index.md index df08b596..e321b138 100644 --- a/content/en/docs/tutorial-ai/Lesson2/_index.md +++ b/content/en/docs/tutorial-ai/Lesson2/_index.md @@ -4,8 +4,6 @@ weight: 10 draft: true --- -# TO BE REVIEWED! - > _Lesson 1_ introduced the first services and a simple chat action. _Lesson 2_ shows how to call an LLM directly and stream its answer through Open Serverless. # Lesson 2 – LLM Access, Secrets & Streaming @@ -24,9 +22,9 @@ You will learn: ## Preparing the Environment -Start from the Mastro GPT starter repository as in the previous lessons. +Start from the MastroGPT starter repository as in the previous lessons. -1. Open the Mastro GPT GitHub organization. +1. Open the MastroGPT GitHub organization. 2. Fork the starter repository into your own account. 3. Create a Codespace from your fork. 4. Log in from the Open Serverless extension using your course credentials. diff --git a/content/en/docs/tutorial-ai/Lesson3/_index.md b/content/en/docs/tutorial-ai/Lesson3/_index.md index 02ad5fb7..c5de7770 100644 --- a/content/en/docs/tutorial-ai/Lesson3/_index.md +++ b/content/en/docs/tutorial-ai/Lesson3/_index.md @@ -4,6 +4,453 @@ weight: 10 draft: true --- -# TO BE RERGANIZED AND REVIEWED! +> _Lesson 2_ introduced LLM access, secrets, and streaming. _Lesson 3_ adds authenticated web actions, structured forms, and custom displays. +# Lesson 3 – Authentication, Forms & Displays +In this lesson you will improve the input and output of your AI applications. + +You will learn: + +- Why Pinocchio actions need an extra authentication check +- How Open Serverless action authentication works +- How to protect public web actions with the Pinocchio login token +- How to return forms from an action +- How to build prompts from form data +- How to return custom display data +- How to build a chess puzzle generator with structured input and visual output + +--- + +## Preparing the Environment + +Before starting Lesson 3, update your fork of the starter repository. + +1. Open your fork on GitHub. +2. If GitHub shows that your fork is behind, click **Sync fork**. +3. Update your fork from the upstream starter. +4. Start a fresh Codespace from your fork. +5. Log in from the Open Serverless extension. +6. Select **Lesson 3** from the lessons panel. + +Lesson 3 is independent from the previous lessons. You can run it even if you do not have the code from Lesson 1 or Lesson 2. + +> If your Codespace contains uncommitted changes, save or commit them before deleting it. + +--- + +## Why Authentication Matters + +Pinocchio itself is protected by a login screen, but the actions invoked by Pinocchio are web actions. + +Web actions must be public so that the browser interface can call them. This means that, by default, anyone with the URL can invoke them. + +For AI applications this is risky because public actions may call LLMs, use user data, or access other services. + +The solution used in this course is a custom authentication check based on the Pinocchio login token. + +--- + +## Open Serverless Action Authentication + +Normal Open Serverless actions are protected by default. + +The CLI stores the authentication key in the local properties file. This key contains: + +| Value | Purpose | +|-----------|------------------------------------------| +| `AUTH` | User and secret used to invoke actions | +| `APIHOST` | Open Serverless API endpoint | + +You can load these values into your shell: + +```bash +source .wskprops +``` + +Protected actions cannot be called directly from a browser URL. They must be invoked with the authentication key and the correct Open Serverless API parameters. + +For example, to get an action URL: + +```bash +ops url mastrogpt/index +``` + +If the command prints an extra status line, you can keep only the URL: + +```bash +ops url mastrogpt/index | tail -n +2 +``` + +You can store it in a variable: + +```bash +URL=$(ops url mastrogpt/index | tail -n +2) +``` + +Then invoke a protected action with authentication: + +```bash +curl -u "$AUTH" -X POST "$URL?blocking=true&result=true" +``` + +This is how normal protected Open Serverless actions are invoked under the hood. + +--- + +## Web Actions Are Public + +Pinocchio actions are deployed as web actions, usually with `--web true`. + +That makes them reachable from the browser, but it also means they are not protected by the normal Open Serverless action key. + +For example: + +```bash +ops ide deploy auth +URL=$(ops url mastrogpt/auth | tail -n +2) +curl "$URL" +``` + +Without an extra check, the action can be called directly. + +The rest of this section shows how to connect these public actions to the Pinocchio login system. + +--- + +## Pinocchio Login Tokens + +When a user logs in to Pinocchio, Pinocchio creates a random token. + +That token is: + +- Stored in a browser cookie +- Sent with each request to the backend actions +- Stored server-side in Redis +- Associated with the logged-in user + +The backend action can verify that the token sent by the browser matches the token stored in Redis. + +If the token is valid, the request came from a logged-in Pinocchio session. If it is missing or invalid, the action should reject the request. + +--- + +## Protecting a Web Action + +Lesson 3 provides an authentication helper that checks whether the current request is authorized. + +The action pattern is: + +```python +def main(args): + if unauthorized(args): + return {"body": {"output": "You are not authenticated"}} + + return {"body": {"output": "You are authenticated"}} +``` + +The `unauthorized` helper reads the token from the request, compares it with Redis, and returns whether the user is logged in. + +After this check: + +- Calling the action from inside Pinocchio succeeds. +- Calling the action directly from its public URL fails. + +Add this check to web actions that should only be used by authenticated Pinocchio users. + +--- + +## Forms in Pinocchio + +Forms let an action request structured input instead of asking the user to type everything in one message. + +A form is a list of field definitions. + +Each field can define: + +| Property | Purpose | +|---------------|----------------------------------------------| +| `name` | Key used when the form is submitted | +| `description` | Text shown to the user | +| `type` | Field type, such as text, textarea, radio | +| `required` | Whether the field must be completed | +| `options` | Choices for radio or checkbox fields | + +Common field types include: + +- `text` +- `textarea` +- `checkbox` +- `radio` +- `file` + +--- + +## Returning a Form + +To show a form, return a response that includes a `form` key. + +Example: + +```python +form = [ + { + "name": "job", + "description": "Your role", + "type": "text", + "required": True, + }, + { + "name": "why", + "description": "Why are you using Apache Open Serverless?", + "type": "textarea", + "required": True, + }, + { + "name": "tone", + "description": "Tone of the post", + "type": "radio", + "options": ["formal", "informal"], + "required": True, + }, +] + +return { + "body": { + "output": "Fill the form to generate a post.", + "form": form, + } +} +``` + +Pinocchio renders the form automatically. + +As in earlier lessons, an action should always return `output`. If the action streams, it should also return `streaming`. + +--- + +## Processing Form Data + +When the user submits a form, the action receives structured data in the input arguments. + +The submitted values can be used to build a prompt. + +Example: + +```python +values = args.get("form", {}) + +job = values.get("job", "") +why = values.get("why", "") +tone = values.get("tone", "formal") + +prompt = f""" +Generate a promotional post for Apache Open Serverless. + +The reader is a {job}. +The reason for using it is: {why}. +The tone must be {tone}. +""" +``` + +This is the core idea of prompt engineering with forms: + +1. Ask the user for structured information. +2. Convert the form values into a precise prompt. +3. Send that prompt to the LLM. +4. Return the generated answer. + +The post generator in this lesson follows this pattern. + +--- + +## Displays in Pinocchio + +Forms improve input. Displays improve output. + +Pinocchio can forward custom response keys to a display action. This lets you show visual content such as: + +- Chessboards +- PDFs +- Diagrams +- Circuits +- Other domain-specific views + +Pinocchio already knows how to handle some standard keys: + +| Key | Purpose | +|----------|----------------------------| +| `output` | Main chat response | +| `state` | Stored conversation state | +| `form` | Form definition | + +Unknown keys are forwarded to the display system. + +For example: + +```python +return { + "body": { + "output": "Here is the chess position.", + "chess": "8/8/8/8/8/8/8/8 w - - 0 1", + } +} +``` + +The `chess` key is not a standard chat key, so Pinocchio forwards it to the display action. The display action can then render a chessboard. + +--- + +## Testing Display Output From the CLI + +You can inspect display data from the command line. + +Invoke a demo action that returns display data: + +```bash +ops ai cli +``` + +Then call the demo with a chess request and save the result: + +```bash +ops action invoke mastrogpt/demo -p input chess --result > chess.json +``` + +The result contains the normal output plus an extra display key such as `chess`. + +That extra data is what Pinocchio forwards to the display action. + +--- + +## Chess Puzzle Generator + +The lesson combines forms, prompt engineering, LLM output parsing, and displays in a chess puzzle generator. + +The flow is: + +1. The user asks for a puzzle. +2. The action asks the LLM to generate a chess puzzle in FEN format. +3. The action extracts the FEN string from the LLM response. +4. The action returns the FEN string under the `chess` key. +5. Pinocchio forwards the `chess` value to the display. +6. The display renders the chessboard. + +FEN is a standard chess notation used to describe a board position. + +Example: + +```text +8/8/8/8/8/8/8/8 w - - 0 1 +``` + +The puzzle generator can extract the FEN string from the LLM output with a regular expression. + +Example pattern: + +```python +import re + +match = re.search(r"([rnbqkpRNBQKP1-8/]+ [wb] [-KQkq]+ [-a-h1-8]+ \d+ \d+)", output) +fen = match.group(1) if match else "" +``` + +Regular expressions are useful for extracting structured fragments from LLM output. You do not need to write them all by hand; an LLM can often help generate or refine the pattern. + +--- + +## Exercise: Add a Puzzle Form + +Modify the puzzle generator so it asks the user what kind of puzzle to create. + +Instead of generating a generic chess puzzle, return a form with piece options such as: + +- Queen +- Rook +- Knight +- Bishop + +The user should be able to select one or more pieces. + +Then build a prompt from the selected values. + +Example prompt: + +```text +Generate a chess puzzle in FEN format. +The puzzle must include a queen and a knight. +Return the FEN string clearly. +``` + +The completed flow should be: + +1. The user writes `puzzle`. +2. Pinocchio displays the piece-selection form. +3. The user selects pieces, for example queen and knight. +4. The action builds a prompt from those choices. +5. The LLM returns a puzzle in FEN format. +6. The action extracts the FEN string. +7. The action returns the `chess` key. +8. Pinocchio displays the board. + +--- + +## Checking the Solution + +Try the exercise before looking at the solution. + +The lesson files include a hidden solution that can be extracted with the course command shown in the video. + +After extracting it, compare your file with the solution using VS Code: + +1. Open your puzzle generator file. +2. Open the solution file. +3. Use **Select for Compare**. +4. Use **Compare with Selected**. + +The final result should display a form first, then generate and render a puzzle that matches the selected pieces. + +--- + +## Deploy and Try It + +Deploy the updated actions: + +```bash +ops ide deploy +``` + +Open Pinocchio and try: + +```text +puzzle +``` + +Select pieces in the form, submit it, and verify that a chessboard appears in the display. + +Also test authentication: + +1. Invoke the action from Pinocchio. +2. Invoke the same public URL directly with `curl`. +3. Confirm that direct access is rejected when the authentication check is enabled. + +--- + +## Summary + +You now: + +- Updated your fork before starting a new lesson +- Learned the difference between protected actions and public web actions +- Protected Pinocchio web actions with a login token check +- Built forms for structured user input +- Converted form data into LLM prompts +- Returned custom display keys from actions +- Used FEN strings to render chess puzzle output +- Started combining authentication, forms, prompt engineering, and displays + +--- + +## Up Next + +Next lessons will continue expanding these application patterns with richer LLM workflows and more complete Open Serverless applications. From 2c65b053fe2ef2925d691dd92c3a8c936ee8f7b5 Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:10:32 +0200 Subject: [PATCH 06/10] fixed #82 lesson 4 --- content/en/docs/tutorial-ai/Lesson4/_index.md | 635 +++++++++++++++++- 1 file changed, 634 insertions(+), 1 deletion(-) diff --git a/content/en/docs/tutorial-ai/Lesson4/_index.md b/content/en/docs/tutorial-ai/Lesson4/_index.md index 632e866e..281b501f 100644 --- a/content/en/docs/tutorial-ai/Lesson4/_index.md +++ b/content/en/docs/tutorial-ai/Lesson4/_index.md @@ -4,4 +4,637 @@ weight: 10 draft: true --- -# TO BE RERGANIZED AND REVIEWED! +> _Lesson 3_ introduced authentication, forms, and custom displays. _Lesson 4_ builds a stateful assistant that remembers the conversation. + +# Lesson 4 – Stateful Assistants with Redis + +In this lesson you will turn a stateless chat into an assistant. + +An assistant is a chat that remembers what happened earlier in the conversation. To do that, it must store conversation history somewhere and reload it on the next request. + +You will learn: + +- How the OpenAI-compatible chat API works +- How message history is represented +- How to stream responses with the OpenAI API +- How to wrap LLM calls in a Python class +- How Redis stores temporary conversation history +- How to return state to Pinocchio +- How to build a stateful assistant + +--- + +## Preparing the Environment + +Start from your fork of the starter repository. + +1. Open your fork on GitHub. +2. If GitHub shows that your fork is behind, click **Sync fork**. +3. Start a Codespace from the updated fork. +4. Log in from the Open Serverless extension. +5. Select **Lesson 4** from the lessons panel. + +Lesson 4 is independent from the previous lessons. You can run it even if your workspace does not contain the code from Lessons 1, 2, or 3. + +--- + +## Stateless Chats vs Assistants + +The chats from the previous lessons are stateless. + +That means each request is handled alone: + +1. The user sends one input. +2. The action sends that input to the LLM. +3. The action returns the answer. +4. The next request starts again from zero. + +An assistant is stateful. + +It keeps the previous messages and sends the full conversation history to the LLM every time. This lets the LLM answer consistently because it can see what was already said. + +--- + +## OpenAI-Compatible APIs + +The OpenAI chat API has become a common interface for LLM applications. + +Many providers implement an API that is fully or partly compatible with it, including Ollama. This means that an application written with the OpenAI client can often switch providers by changing: + +- The base URL +- The API key +- The model name + +For this course, the OpenAI client is used to call the Ollama-compatible endpoint. + +--- + +## Creating an OpenAI Client + +Open the interactive CLI: + +```bash +ops ai cli +``` + +Create a client: + +```python +import os +from openai import OpenAI + +host = os.getenv("OLLAMA_HOST") +auth = os.getenv("AUTH") + +base_url = f"https://{auth}@{host}/v1" +api_key = "unused" + +client = OpenAI( + base_url=base_url, + api_key=api_key, +) +``` + +With Ollama in this environment, authentication is carried by the URL. The `api_key` value is still required by the OpenAI client, but it is not the value used for authentication here. + +With other providers, both `base_url` and `api_key` are usually meaningful. + +--- + +## Chat Messages + +OpenAI-compatible chat requests use a list of messages. + +Each message has: + +| Field | Purpose | +|-----------|---------------------------------| +| `role` | Who wrote the message | +| `content` | Text content of the message | + +Important roles are: + +| Role | Meaning | +|-------------|----------------------------------------------| +| `system` | Instructions that configure the assistant | +| `user` | User input | +| `assistant` | Previous LLM answers | + +The LLM receives the full list and produces the next assistant message. + +Example: + +```python +messages = [ + { + "role": "user", + "content": "What is the capital of Italy?", + } +] +``` + +--- + +## Calling the Chat API + +Send the message list to the model: + +```python +res = client.chat.completions.create( + model="llama3.1:8b", + messages=messages, +) + +answer = res.choices[0].message.content +print(answer) +``` + +The response is structured. It can contain multiple choices, but most applications use the first one: + +```python +res.choices[0].message.content +``` + +This is the basic pattern for calling the OpenAI-compatible chat API. + +--- + +## Streaming With the Chat API + +To stream the answer, add `stream=True`: + +```python +stream = client.chat.completions.create( + model="llama3.1:8b", + messages=messages, + stream=True, +) +``` + +The result is a Python iterator. + +Each item contains a small piece of the answer, called a delta: + +```python +for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + print(delta, end="") +``` + +This is the OpenAI-compatible version of the streaming behavior introduced in Lesson 2. + +--- + +## Python Classes: Quick Reminder + +Lesson 4 uses Python classes to wrap the chat logic. + +A class defines an object with data and methods. + +Example: + +```python +class Counter: + def __init__(self, start=0): + self.value = start + + def count(self): + current = self.value + self.value += 1 + return current +``` + +Use it like this: + +```python +c = Counter() +print(c.count()) +print(c.count()) +``` + +Important Python class rules: + +- `__init__` is the constructor. +- Each method receives `self` as its first argument. +- Object fields are usually created with `self.name = value`. +- Object fields are read with `self.name`. + +The rest of the lesson assumes you already know the basic idea of object-oriented programming. + +--- + +## Chat Wrapper Class + +The lesson introduces a class that wraps the OpenAI client and keeps a local message list. + +The class responsibilities are: + +1. Create the OpenAI client. +2. Store the message history. +3. Add new messages. +4. Send the history to the LLM. +5. Add the assistant response back to the history. + +Conceptually: + +```python +class Chat: + def __init__(self): + self.client = OpenAI(...) + self.messages = [] + + def add(self, text): + role, content = text.split(":", 1) + self.messages.append({ + "role": role.strip(), + "content": content.strip(), + }) + + def complete(self): + res = self.client.chat.completions.create( + model="llama3.1:8b", + messages=self.messages, + ) + answer = res.choices[0].message.content + self.messages.append({ + "role": "assistant", + "content": answer, + }) + return answer +``` + +The class lets you write a sequence like: + +```python +chat = Chat() +chat.add("system: When I give you a country, answer with its capital.") +chat.complete() + +chat.add("user: Italy") +print(chat.complete()) + +chat.add("user: France") +print(chat.complete()) +``` + +Because the class keeps the message history, the model remembers the system instruction. + +--- + +## Exercise 1: Add Streaming to the Chat Class + +Find: + +```text +TODO E4.1 +``` + +The first exercise is to make the chat class stream its answers. + +You need to: + +1. Add a streaming function. +2. Save the action arguments when the class is initialized. +3. Request `stream=True` when calling the chat API. +4. Read each streamed delta from the OpenAI response. +5. Send each delta to the Open Serverless stream socket. +6. Collect the complete answer. +7. Save the assistant answer in the message history. +8. Return `streaming: True` from the action response. + +The important OpenAI streaming extraction is: + +```python +for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + # send delta to the stream + # append delta to the final result +``` + +After this exercise, the chat should answer progressively instead of waiting for the whole response before displaying anything. + +To download the solution, use the course command shown in the lesson: + +```bash +ops ai lesson 4 assistant --solution +``` + +--- + +## Why We Still Need State + +The chat class keeps history only while the current action invocation is running. + +After the action finishes, that memory disappears. + +To build a real assistant, you need to store the history somewhere outside the action invocation. Lesson 4 uses Redis for that. + +--- + +## Redis Overview + +Redis means Remote Dictionary Server. + +It is a fast data store used by server applications to keep shared state. + +Redis can store several data types: + +| Type | Use case | +|-------------|-------------------------------------------| +| String | Single value | +| List | Ordered sequence of values | +| Hash | Field/value table | +| Set | Unordered unique values | +| Sorted set | Ordered unique values with scores | +| Stream | Event-like append-only data | + +For this lesson, you will use Redis lists to store chat messages. + +--- + +## Redis Prefixes + +In this course environment, Redis is shared. + +Users cannot read each other's values, but keys may be visible. For that reason, each user must use the assigned key prefix. + +The Redis connection values are available in the environment. + +Example: + +```python +import os +import redis + +prefix = os.getenv("REDIS_PREFIX") +url = os.getenv("REDIS_URL") + +rd = redis.from_url(url) +``` + +Always include the prefix when creating keys. + +--- + +## Redis Strings and Lists + +Store and read a single value: + +```python +rd.set(f"{prefix}:example", "hello") +value = rd.get(f"{prefix}:example") +print(value.decode("utf-8")) +``` + +Redis values are bytes, so decode them when you need strings. + +Store values in a list: + +```python +key = f"{prefix}:messages" + +rd.rpush(key, "first") +rd.rpush(key, "second") + +items = rd.lrange(key, 0, -1) +for item in items: + print(item.decode("utf-8")) +``` + +Lists are a good fit for conversation history because messages have a natural order. + +--- + +## History Class + +The lesson introduces a `History` class that stores message history in Redis. + +The class does three important things: + +1. Creates a unique conversation ID when a new chat starts. +2. Saves messages under a Redis key based on that ID. +3. Reloads messages when the conversation continues. + +A unique ID can be created with Python's `uuid` library: + +```python +import uuid + +conversation_id = str(uuid.uuid4()) +``` + +Redis keys are set to expire after one day so old conversations are cleaned automatically. + +Conceptually: + +```python +class History: + def __init__(self, conversation_id=None): + self.conversation_id = conversation_id or str(uuid.uuid4()) + self.key = f"{prefix}:history:{self.conversation_id}" + + def id(self): + return self.conversation_id + + def save(self, message): + rd.rpush(self.key, json.dumps(message)) + rd.expire(self.key, 60 * 60 * 24) + + def load(self): + items = rd.lrange(self.key, 0, -1) + return [json.loads(item.decode("utf-8")) for item in items] +``` + +This creates temporary state: the assistant remembers the conversation for a day, then Redis deletes it. + +--- + +## Loading History Into a Chat + +The `History` class and the `Chat` class work together. + +Example flow: + +```python +history = History() + +history.save({ + "role": "system", + "content": "When I give you a country, answer with its capital.", +}) + +conversation_id = history.id() +``` + +Later, reload the same history: + +```python +history = History(conversation_id) +messages = history.load() + +chat = Chat() +chat.messages = messages +``` + +Now the chat can continue from the previous conversation. + +--- + +## Building the Stateful Assistant + +The assistant action should: + +1. Read the current `state` from `args`. +2. Create a `History` object using that state. +3. Create a `Chat`. +4. Load the history into the chat. +5. Add the new user input. +6. Save the user input to Redis. +7. Complete the chat. +8. Save the assistant response to Redis. +9. Return the conversation ID as `state`. + +Conceptually: + +```python +def main(args): + inp = args.get("input", "") + state = args.get("state") + + history = History(state) + chat = Chat(args) + chat.messages = history.load() + + user_message = { + "role": "user", + "content": inp, + } + + chat.messages.append(user_message) + history.save(user_message) + + answer = chat.complete() + + history.save({ + "role": "assistant", + "content": answer, + }) + + return { + "body": { + "output": answer, + "state": history.id(), + } + } +``` + +The key part is returning `state`. Pinocchio sends that state back on the next request, so the action can reload the same Redis history. + +--- + +## Exercise 2: Turn the Chat Into an Assistant + +Find: + +```text +TODO E4.2 +``` + +Transform the stateless chat into a stateful assistant. + +You need to: + +1. Read the previous conversation ID from `state`. +2. Load the history from Redis. +3. Initialize the chat with the loaded messages. +4. Save the user's new message. +5. Complete the chat. +6. Save the assistant response. +7. Return the conversation ID as `state`. + +When this works, the assistant should remember instructions from earlier messages. + +For example: + +```text +When I give you a country, answer with its capital. +``` + +Then: + +```text +Italy +``` + +The assistant should answer: + +```text +Rome +``` + +Then: + +```text +France +``` + +The assistant should remember the instruction and answer: + +```text +Paris +``` + +--- + +## Deploy and Try It + +Deploy the updated lesson actions: + +```bash +ops ide deploy +``` + +Open Pinocchio and select the stateful assistant. + +Test the assistant with a sequence: + +```text +When I give you a country, answer with its capital. +``` + +```text +Italy +``` + +```text +France +``` + +```text +United States +``` + +If the assistant remembers the instruction, the Redis-backed state is working. + +--- + +## Summary + +You now: + +- Used the OpenAI-compatible API with Ollama +- Built chat requests from role-based message lists +- Streamed responses from the OpenAI client +- Wrapped chat behavior in a Python class +- Used Redis strings and lists +- Stored conversation history with a unique ID +- Returned `state` so Pinocchio can continue a conversation +- Built a stateful assistant that remembers previous messages + +--- + +## Up Next + +Next lessons will build on stateful assistants to create richer AI applications that combine memory, tools, and structured workflows. From 1eea108d23ce995099e38e8ab717661bafb14d9c Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:25:17 +0200 Subject: [PATCH 07/10] fixed #83 fixed #84 fixed #85 lessons 5 6 7 --- content/en/docs/tutorial-ai/Lesson5/_index.md | 431 +++++++++++++++ content/en/docs/tutorial-ai/Lesson6/_index.md | 471 ++++++++++++++++ content/en/docs/tutorial-ai/Lesson7/_index.md | 509 +++++++++++++++++- 3 files changed, 1410 insertions(+), 1 deletion(-) diff --git a/content/en/docs/tutorial-ai/Lesson5/_index.md b/content/en/docs/tutorial-ai/Lesson5/_index.md index 75b8d4bf..da275f4e 100644 --- a/content/en/docs/tutorial-ai/Lesson5/_index.md +++ b/content/en/docs/tutorial-ai/Lesson5/_index.md @@ -6,4 +6,435 @@ draft: true # TO BE RERGANIZED AND REVIEWED! +> _Lesson 4_ introduced stateful assistants with Redis. _Lesson 5_ adds computer vision and object storage. +# Lesson 5 - Vision Models & S3 Storage + +In this lesson you will analyze images with an open source vision model and store uploaded files in S3-compatible object storage. + +You will learn: + +- How to send images to an Ollama vision model +- How to upload images with Pinocchio forms +- How to display base64 images in HTML +- How S3-compatible storage works in Open Serverless +- How to read, write, list, delete, and display stored objects +- How to generate signed URLs for temporary public access + +--- + +## Preparing the Environment + +Start from your starter Codespace as usual. + +1. Open your fork of the starter repository. +2. Sync the fork if updates are available. +3. Start the Codespace. +4. Log in from the Open Serverless extension. +5. Select **Lesson 5** from the lessons panel. + +Lesson 5 is independent from the previous lessons. + +--- + +## Vision With Ollama + +To analyze an image, use a model that supports vision, such as the Llama vision model used in this lesson. + +The image must be sent as base64 text because the request body is JSON. + +The basic flow is: + +1. Read the image file. +2. Encode it as base64. +3. Build a chat message with a prompt and an `images` field. +4. Send the request to Ollama. +5. Collect the streamed response. + +Example: + +```python +import base64 +import os +import requests + +host = os.getenv("OLLAMA_HOST") +auth = os.getenv("AUTH") +url = f"https://{auth}@{host}/api/chat" + +with open("cat.jpg", "rb") as f: + image = base64.b64encode(f.read()).decode("utf-8") + +msg = { + "model": "llama3.2-vision", + "messages": [ + { + "role": "user", + "content": "What is in this image?", + "images": [image], + } + ], +} + +res = requests.post(url, json=msg, stream=True) +``` + +The result is streamed. You need to iterate over the response and collect the pieces. + +--- + +## Vision Action + +The lesson wraps the vision call in a class. + +The class responsibilities are: + +- Read Ollama access values from `args` or environment variables +- Build the vision request +- Send the image to the model +- Collect the streamed response +- Return the final description + +Conceptually: + +```python +class Vision: + def __init__(self, args): + self.host = args.get("OLLAMA_HOST", os.getenv("OLLAMA_HOST")) + self.auth = args.get("AUTH", os.getenv("AUTH")) + self.url = f"https://{self.auth}@{self.host}/api/chat" + + def decode(self, image): + msg = { + "model": "llama3.2-vision", + "messages": [ + { + "role": "user", + "content": "What is in this image?", + "images": [image], + } + ], + } + res = requests.post(self.url, json=msg, stream=True) + return collect(res) +``` + +For simplicity, this lesson collects the streamed vision result before returning it. + +--- + +## Uploading Images With a Form + +Pinocchio forms can include file fields. + +Example form field: + +```python +form = [ + { + "name": "pic", + "description": "Choose an image", + "type": "file", + "required": True, + } +] +``` + +Return the form when there is no submitted data: + +```python +return { + "body": { + "output": "Upload an image to analyze.", + "form": form, + } +} +``` + +When the form is submitted, the uploaded file is available as base64 data. + +Example: + +```python +values = args.get("form", {}) +image = values.get("pic", "") +``` + +Pass that base64 image to the vision class. + +--- + +## Displaying the Uploaded Image + +Pinocchio can display HTML returned by an action. + +To display a base64 image, create an HTML image tag: + +```python +html = f'' +``` + +Return the description and the HTML display data: + +```python +return { + "body": { + "output": description, + "html": html, + } +} +``` + +The `html` key is forwarded to the display system, so the user can see both the uploaded image and the LLM analysis. + +--- + +## S3-Compatible Storage + +Open Serverless provides object storage through an S3-compatible API. + +S3 stores files as objects inside buckets. + +Important concepts: + +| Concept | Meaning | +|----------|------------------------------------------| +| Endpoint | URL used to access S3 | +| Bucket | Storage area for objects | +| Key | Object name or path inside the bucket | +| Object | Stored file content | + +This lesson uses S3 to save uploaded images and analyze them later. + +--- + +## Three S3 Endpoints + +There are three S3 access contexts to keep separate: + +| Endpoint type | Used for | +|---------------|-----------------------------------------------| +| Internal | Deployed actions in production | +| Test | Local tests inside the course environment | +| External | Public display URLs for browser access | + +The internal endpoint is used by deployed actions and is configured with variables such as: + +- `S3_HOST` +- `S3_PORT` +- `S3_BUCKET` +- `S3_ACCESS_KEY` +- `S3_SECRET_KEY` + +Tests use values from the test environment. + +When an image must be displayed in the browser, use the external URL. + +--- + +## Creating an S3 Client + +Use the S3 endpoint, key, secret, bucket, and region to create a client. + +The Open Serverless S3 service is compatible with the AWS S3 API, so the default region can be `us-east-1`. + +Example: + +```python +import boto3 +import os + +endpoint = f"http://{os.getenv('S3_HOST')}:{os.getenv('S3_PORT')}" +bucket = os.getenv("S3_BUCKET") + +s3 = boto3.client( + "s3", + endpoint_url=endpoint, + aws_access_key_id=os.getenv("S3_ACCESS_KEY"), + aws_secret_access_key=os.getenv("S3_SECRET_KEY"), + region_name="us-east-1", +) +``` + +--- + +## Reading and Writing Objects + +Write an object: + +```python +s3.put_object( + Bucket=bucket, + Key="cat.jpg", + Body=data, +) +``` + +Read it back: + +```python +res = s3.get_object(Bucket=bucket, Key="cat.jpg") +data = res["Body"].read() +``` + +List objects: + +```python +res = s3.list_objects_v2(Bucket=bucket) +contents = res.get("Contents", []) +``` + +Delete an object: + +```python +s3.delete_object(Bucket=bucket, Key="cat.jpg") +``` + +These are the core operations used by the lesson storage action. + +--- + +## Signed URLs + +Stored objects are private by default. + +To show an image in the browser, generate a signed URL. A signed URL grants temporary public access to a single object. + +Example: + +```python +url = s3.generate_presigned_url( + "get_object", + Params={ + "Bucket": bucket, + "Key": key, + }, + ExpiresIn=3600, +) +``` + +The URL is valid for one hour. + +Because signed URLs may be generated with the internal S3 endpoint, the lesson rewrites them to use the external S3 URL before returning them to the browser. + +--- + +## Bucket Helper Class + +The lesson wraps S3 access in a bucket class. + +The class provides methods to: + +- Write an object +- Read an object +- Remove objects by prefix +- Search objects by prefix or substring +- Get object size +- Generate an external signed URL + +This keeps action code small and reusable. + +--- + +## Storage Action + +The storage action lets you manage uploaded images from Pinocchio. + +The input commands are: + +| Input prefix | Behavior | +|--------------|----------------------------------| +| `*` | Search/list stored objects | +| `!` | Remove objects by prefix | +| `@` | Analyze an existing stored image | + +Example: + +```text +* +``` + +lists stored files. + +Example: + +```text +@cat +``` + +finds a matching image, reads it from S3, sends it to the vision model, and returns both the analysis and an image display. + +Example: + +```text +!cat +``` + +removes matching objects with that prefix. + +--- + +## Exercise: Save Form Uploads to S3 + +Combine the two examples from the lesson. + +Modify the form-based image analyzer so that it: + +1. Shows a file upload form. +2. Receives the uploaded image. +3. Saves the image to S3. +4. Reads or uses the saved image. +5. Sends the image to the vision model. +6. Generates an external signed URL. +7. Displays the image using the external URL. +8. Returns the LLM description. + +This gives you a complete upload, store, analyze, and display workflow. + +--- + +## Deploy and Try It + +Deploy the lesson actions: + +```bash +ops ide deploy +``` + +Open Pinocchio and try the vision form. + +Then try the storage action: + +```text +* +``` + +```text +@cat +``` + +```text +!cat +``` + +Verify that images can be uploaded, listed, analyzed, displayed, and removed. + +--- + +## Summary + +You now: + +- Sent base64 images to a vision model +- Built a file upload form +- Displayed uploaded images with HTML +- Used S3-compatible storage from Open Serverless +- Read, wrote, listed, and deleted objects +- Generated signed URLs for temporary public access +- Combined vision and storage into an image workflow + +--- + +## Up Next + +Next lesson introduces vector databases and embeddings, which are the foundation for retrieval augmented generation. diff --git a/content/en/docs/tutorial-ai/Lesson6/_index.md b/content/en/docs/tutorial-ai/Lesson6/_index.md index 69e9f1c0..8cd9af70 100644 --- a/content/en/docs/tutorial-ai/Lesson6/_index.md +++ b/content/en/docs/tutorial-ai/Lesson6/_index.md @@ -5,3 +5,474 @@ draft: true --- # TO BE RERGANIZED AND REVIEWED! + +> _Lesson 5_ introduced vision models and S3 storage. _Lesson 6_ introduces embeddings and vector databases. + +# Lesson 6 - Embeddings & Vector Databases + +In this lesson you will store text in a vector database and search it by semantic similarity. + +You will learn: + +- What embeddings are +- Why vector databases are useful for LLM applications +- How to create a Milvus collection +- How to insert embedded text +- How to search by vector similarity +- How to load text from PDF files +- How document loading prepares data for RAG + +--- + +## Preparing the Environment + +Start from your fork of the starter repository. + +1. Sync your fork if updates are available. +2. Start your Codespace. +3. Log in from the Open Serverless extension. +4. Select **Lesson 6** from the lessons panel. + +--- + +## Vector Databases + +A vector database stores numerical representations of data and searches by similarity. + +In this course, the vector database is **Milvus**. + +Vector search is important for LLM applications because it lets you find text that is semantically related to a question. + +For example: + +- `cat` is closer to `dog` than to `orange` +- `bitcoin` is closer to `blockchain` than to `banana` + +This similarity search is the foundation of RAG: retrieval augmented generation. + +--- + +## Milvus Concepts + +Milvus is organized into databases and collections. + +You can think of a collection like a table. + +Each collection has: + +| Concept | Meaning | +|------------|------------------------------------------| +| Schema | Field structure for stored records | +| Field | A value such as text or an embedding | +| Entity | One stored record | +| Index | Data structure that speeds up search | +| Metric | Distance function used for similarity | + +For text search, each record usually stores: + +- The original text +- The embedding vector for that text + +--- + +## Connecting to Milvus + +Create a Milvus client from the environment values. + +```python +import os +from pymilvus import MilvusClient + +uri = os.getenv("MILVUS_HOST") +token = os.getenv("MILVUS_TOKEN") +database = os.getenv("MILVUS_DB") + +client = MilvusClient( + uri=uri, + token=token, + db_name=database, +) +``` + +List collections: + +```python +client.list_collections() +``` + +Drop a collection while experimenting: + +```python +client.drop_collection("test") +``` + +--- + +## Creating a Collection + +Choose a vector dimension that matches the embedding model. + +In the lesson, the embedding size is `1024`. + +Create a schema with: + +- An auto-generated ID +- A text field +- A vector field + +Conceptually: + +```python +from pymilvus import DataType + +collection = "test" +dim = 1024 + +schema = MilvusClient.create_schema(auto_id=True) + +schema.add_field( + field_name="id", + datatype=DataType.INT64, + is_primary=True, +) + +schema.add_field( + field_name="text", + datatype=DataType.VARCHAR, + max_length=4096, +) + +schema.add_field( + field_name="embedding", + datatype=DataType.FLOAT_VECTOR, + dim=dim, +) +``` + +Then create an index for the vector field: + +```python +index_params = client.prepare_index_params() + +index_params.add_index( + field_name="embedding", + index_type="AUTOINDEX", + metric_type="IP", +) +``` + +Create the collection: + +```python +client.create_collection( + collection_name=collection, + schema=schema, + index_params=index_params, +) +``` + +--- + +## Inserting Records + +Each record needs text and an embedding. + +Before learning embeddings, you can insert a fake vector only to understand the Milvus API: + +```python +data = [ + { + "text": "hello world", + "embedding": [0.0] * 1024, + } +] + +client.insert( + collection_name="test", + data=data, +) +``` + +Query records: + +```python +res = client.query( + collection_name="test", + filter="", + output_fields=["text"], +) +``` + +This proves that the collection can store and return records. For real search, use embeddings. + +--- + +## Embeddings + +An embedding is a numerical vector that represents text semantically. + +Instead of returning text, an embedding model returns a list of numbers. + +That list can be stored in Milvus and used for similarity search. + +Create an embedding request to Ollama: + +```python +import os +import requests + +host = os.getenv("OLLAMA_HOST") +auth = os.getenv("AUTH") +url = f"https://{auth}@{host}/api/embeddings" + +msg = { + "model": "mxbai-embed-large", + "prompt": "hello world", +} + +res = requests.post(url, json=msg).json() +embedding = res["embedding"] +``` + +The embedding model must produce vectors with the same dimension as the Milvus collection. + +--- + +## Vector DB Helper Class + +The lesson wraps Milvus and embedding logic in a `VectorDB` class. + +The class can: + +- Create or reset a collection +- Embed text with Ollama +- Insert text and embedding together +- Search for semantically similar text +- Remove records + +Conceptually: + +```python +class VectorDB: + def embed(self, text): + res = requests.post(self.embed_url, json={ + "model": self.embed_model, + "prompt": text, + }).json() + return res["embedding"] + + def insert(self, text): + embedding = self.embed(text) + self.client.insert( + collection_name=self.collection, + data=[{ + "text": text, + "embedding": embedding, + }], + ) +``` + +--- + +## Searching by Similarity + +To search, embed the search text and compare it against stored embeddings. + +Example: + +```python +query = "test" +query_embedding = vdb.embed(query) + +res = client.search( + collection_name="test", + data=[query_embedding], + anns_field="embedding", + limit=5, + output_fields=["text"], +) +``` + +The closest records are returned first. + +If the collection contains: + +```text +This is a test +This is another test +Testing the system +Hello world +``` + +then searching for: + +```text +test +``` + +should return the test-related sentences before unrelated text. + +--- + +## Loader Action + +The lesson provides an action for loading and searching text. + +The input commands are: + +| Input prefix | Behavior | +|--------------|-----------------------------| +| text | Insert text into Milvus | +| `*text` | Search for related text | +| `!text` | Remove matching records | + +Examples: + +```text +This is a test +``` + +```text +*test +``` + +```text +!test +``` + +This action can be used from Pinocchio or from the command line. + +--- + +## Loading Text From PDFs + +Typing text manually is useful for tests, but real databases are loaded from documents. + +The lesson uses PDF extraction to import document text. + +The basic PDF flow is: + +1. Open the PDF. +2. Extract text from each page. +3. Split the text into smaller sentences or chunks. +4. Send each piece to the loader action. + +Example libraries: + +- `pymupdf` for reading PDFs +- a sentence tokenizer for splitting text + +Small pieces work better than huge pages because embeddings are more useful when each record has focused content. + +--- + +## `ops ai loader` + +The course plugin includes a loader command that automates document import. + +The loader can: + +- Convert a PDF into text +- Split the text into chunks or sentences +- Invoke an action for each piece of text + +Example: + +```bash +ops ai loader lessons/bitcoin.pdf --action vdb-load +``` + +The loader sends each extracted sentence or chunk to the `vdb-load` action, which embeds it and stores it in Milvus. + +After loading the Bitcoin white paper, search for terms such as: + +```text +*bitcoin +``` + +```text +*merkle +``` + +The vector database should return related passages. + +--- + +## Exercise: Import Web Pages + +Extend the loader so it can import content from a web URL. + +If the input starts with: + +```text +http +``` + +the loader should: + +1. Download the web page. +2. Extract readable text from the HTML. +3. Split the text into chunks. +4. Send each chunk to the vector database loader action. + +Recommended tools: + +- `requests` to fetch the page +- `BeautifulSoup` to extract text +- regular expressions to split text into useful chunks + +Use lighter splitting for web pages when a full tokenizer is too heavy. + +--- + +## Deploy and Try It + +Deploy the lesson actions: + +```bash +ops ide deploy +``` + +Insert text: + +```text +This is a test +``` + +Search: + +```text +*test +``` + +Load a PDF: + +```bash +ops ai loader lessons/bitcoin.pdf --action vdb-load +``` + +Then search: + +```text +*bitcoin +``` + +--- + +## Summary + +You now: + +- Created a Milvus collection +- Added text and vector fields +- Generated embeddings with Ollama +- Inserted embedded text into Milvus +- Performed vector similarity searches +- Loaded PDF text into the vector database +- Prepared the foundation for RAG + +--- + +## Up Next + +Next lesson combines vector search with LLM generation to build a complete RAG system. diff --git a/content/en/docs/tutorial-ai/Lesson7/_index.md b/content/en/docs/tutorial-ai/Lesson7/_index.md index dabb4211..b479a30f 100644 --- a/content/en/docs/tutorial-ai/Lesson7/_index.md +++ b/content/en/docs/tutorial-ai/Lesson7/_index.md @@ -4,5 +4,512 @@ weight: 10 draft: true --- -\ # TO BE RERGANIZED AND REVIEWED! + +> _Lesson 6_ introduced embeddings and vector search. _Lesson 7_ combines them with LLM generation to build a RAG system. + +# Lesson 7 - Retrieval Augmented Generation + +In this final lesson you will build a RAG system: retrieval augmented generation. + +RAG lets an LLM answer using private or external information by retrieving relevant context from a vector database and adding it to the prompt. + +You will learn: + +- What RAG is +- How vector search creates context +- How chunk size affects retrieval quality +- How to load multiple document collections +- How to query different collections and models +- How the RAG loader and RAG query actions work +- How to design an image RAG exercise + +--- + +## Preparing the Environment + +Start from a clean and updated Codespace. + +1. Commit or save your local changes. +2. Sync your fork if updates are available. +3. Delete and recreate the Codespace if the starter changed significantly. +4. Log in from the Open Serverless extension. +5. Select **Lesson 7** from the lessons panel. + +Lesson 7 contains more complete examples than the previous lessons. It brings together the work from the whole course. + +--- + +## What RAG Means + +RAG means retrieval augmented generation. + +The system works in two phases: + +1. **Retrieval**: search a knowledge source for content related to the user question. +2. **Generation**: send the question plus the retrieved context to the LLM. + +The LLM does not need to already know the information. It can answer using the context provided with the request. + +This is especially useful for private data, internal documents, personal datasets, or new information not present in the model training data. + +--- + +## RAG Flow + +A typical RAG request works like this: + +1. The user asks a question. +2. The question is converted into an embedding. +3. The vector database searches for similar chunks. +4. The retrieved chunks are combined into a context. +5. The context and question are sent to the LLM. +6. The LLM answers using that context. + +Conceptually: + +```text +Question -> Embedding -> Vector search -> Context -> LLM prompt -> Answer +``` + +--- + +## Simple RAG Example + +Start with a small vector database containing facts: + +```text +Lisa is the first daughter of Steve Jobs. +Her name is Lisa Brennan. +The Apple Lisa was named after Lisa. +The name Lisa may also refer to the Mona Lisa. +``` + +Search for: + +```text +Lisa +``` + +The vector database returns related facts. + +Build a context: + +```text +Consider this information: +Lisa is the first daughter of Steve Jobs. +Her name is Lisa Brennan. +The Apple Lisa was named after Lisa. +The name Lisa may also refer to the Mona Lisa. +``` + +Now ask: + +```text +Who is Lisa? +``` + +Without context, the LLM may answer generically. With context, it can answer about Steve Jobs' daughter and the Apple Lisa. + +That is the core RAG mechanism. + +--- + +## Chunk Size + +RAG works best when documents are split into useful chunks. + +Chunks should be: + +- Small enough to search accurately +- Large enough to preserve meaning +- Semantically related where possible + +As a practical starting point, this lesson uses chunks of about 4000 characters, roughly 500 to 800 words. + +This is not perfect, but it is simple and works well enough for learning. + +Better systems often split by: + +- Sections +- Paragraphs +- Headings +- Semantic boundaries + +Good chunking is a major part of content engineering for RAG. + +--- + +## Context Size + +Each LLM has a context window limit. + +The model used in the course can handle a large context, but the request still needs room for: + +- The system prompt +- Retrieved chunks +- The user question +- The answer + +If each chunk is around 4000 characters, around 30 chunks is a practical upper limit for the examples. + +Too much context can make responses slower, more expensive, or less focused. + +--- + +## RAG Loader Action + +The lesson includes a loader action for managing vector database content. + +The loader supports: + +- Multiple collections +- Manual text insertion +- Vector searches +- Record deletion +- Collection deletion +- Search limit changes + +Collections let you keep different knowledge sources separate. + +For example: + +- `bitcoin` +- `linkedin` +- `jobs` + +--- + +## Loader Commands + +The loader action uses short command prefixes. + +| Command | Behavior | +|---------------|---------------------------------------| +| text | Insert text into the current collection | +| `*query` | Search current collection | +| `!text` | Delete records matching text | +| `!!name` | Delete a collection | +| `@name` | Select a collection | +| `#number` | Change search result limit | + +Examples: + +```text +@bitcoin +``` + +selects the `bitcoin` collection. + +```text +This is a test +``` + +adds text to the current collection. + +```text +*bitcoin +``` + +searches for related chunks. + +```text +!test +``` + +removes matching records. + +```text +!!bitcoin +``` + +removes the `bitcoin` collection. + +--- + +## Loading PDFs + +The command-line loader can import documents into a collection. + +Example: + +```bash +ops ai loader --action rag-loader --collection bitcoin lessons/bitcoin.pdf +``` + +The loader: + +1. Extracts text from the PDF. +2. Splits the text into chunks. +3. Sends each chunk to the loader action. +4. Embeds each chunk. +5. Stores each embedded chunk in the selected collection. + +After loading, select the collection: + +```text +@bitcoin +``` + +Search: + +```text +*bitcoin +``` + +```text +*merkle +``` + +--- + +## Private Data Example + +Public documents are often already known by large LLMs. + +RAG becomes more useful with private or local data. The lesson shows this with exported LinkedIn connections. + +The workflow is: + +1. Export contacts from LinkedIn as CSV. +2. Convert the CSV into a more readable text format. +3. Import the text into a vector collection. +4. Ask questions about the imported contacts. + +Example collection: + +```text +linkedin +``` + +Example questions: + +```text +List contacts with the role of CEO. +``` + +```text +List contacts who work in software engineering. +``` + +The example is intentionally simple. Better results require better data structuring and chunking. + +--- + +## RAG Query Action + +The RAG query action combines vector search with an LLM prompt. + +It supports: + +- Multiple collections +- Multiple LLMs +- Variable context size +- Abbreviated collection names +- Streaming responses + +The action parses a short command prefix, performs vector search, builds a context, and sends the question to the selected LLM. + +--- + +## Query Syntax + +Use `@` to choose model, context size, and collection. + +The exact parser is implemented in the lesson code, but the idea is: + +```text +@ question +``` + +Examples: + +```text +@jobs Who is Lisa? +``` + +asks using the `jobs` collection. + +```text +@phi jobs Who is Lisa? +``` + +asks with the Phi model and the `jobs` collection. + +```text +@l List all contacts with the role of CEO. +``` + +uses the collection whose name starts with `l`, such as `linkedin`. + +Short prefixes make it quick to switch collections and models from the chat. + +--- + +## Building the Prompt + +After retrieving chunks, the action builds a prompt like this: + +```text +Use the following context to answer the question. + +Context: + + +Question: + +``` + +Then it sends the prompt to the selected LLM. + +Most of the implementation reuses pieces from earlier lessons: + +- LLM access +- Streaming +- Vector database search +- State and command parsing + +Lesson 7 is mainly about putting those pieces together. + +--- + +## Comparing Models + +The RAG query action can switch models, for example: + +- Llama +- Phi +- Mistral +- DeepSeek + +Different models may answer the same context differently. + +Trying multiple models helps you understand how: + +- Context affects answers +- Smaller models behave with retrieved data +- Prompt wording changes results +- Private datasets need careful structuring + +--- + +## Exercise: Image RAG + +The final exercise is to build a RAG system for images. + +Use what you learned in Lessons 5, 6, and 7. + +The system should: + +1. Upload images from a form or from S3. +2. Use a vision model to describe each image. +3. Store the image description in the vector database. +4. Keep a reference to the original image, such as an S3 key or signed URL. +5. Search images by semantic description. +6. Return matching images and descriptions. + +Example: + +```text +Find images with a bird on a branch. +``` + +The system should retrieve image descriptions related to the prompt and display the matching images. + +--- + +## Suggested Image RAG Design + +A useful record structure could include: + +| Field | Purpose | +|---------------|--------------------------------------| +| `description` | Text generated by the vision model | +| `embedding` | Vector embedding of the description | +| `s3_key` | Stored image key | +| `filename` | Original file name | + +The flow: + +1. Upload an image. +2. Store it in S3. +3. Generate a description with the vision model. +4. Embed the description. +5. Store the description, embedding, and S3 key in Milvus. +6. Search by text query. +7. Generate signed URLs for matching images. +8. Display the images in Pinocchio. + +This combines the main ideas from the course into one project. + +--- + +## Deploy and Try It + +Deploy the lesson actions: + +```bash +ops ide deploy +``` + +Try the loader: + +```text +@jobs +``` + +```text +*Lisa +``` + +Try the RAG query: + +```text +@jobs Who is Lisa? +``` + +Load a PDF: + +```bash +ops ai loader --action rag-loader --collection bitcoin lessons/bitcoin.pdf +``` + +Then query it: + +```text +@bitcoin What is a Merkle tree? +``` + +--- + +## Summary + +You now: + +- Built the mental model of RAG +- Used vector search to retrieve context +- Loaded documents into named collections +- Queried private and document-based data +- Switched collections and LLMs from chat commands +- Saw how chunking affects answer quality +- Designed a final image RAG exercise + +--- + +## Course Wrap-Up + +This first edition of the course introduced the main building blocks for private AI applications with Apache Open Serverless: + +- LLM calls +- Streaming +- Secrets +- Forms +- Displays +- Stateful assistants +- Vision models +- S3 storage +- Embeddings +- Vector databases +- RAG + +From here, you can extend these examples into your own private AI workflows and open source projects. From 90d2810b402899bc8e4ca387a65ac73a754110eb Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:26:06 +0200 Subject: [PATCH 08/10] fixed #83 fixed #84 fixed #85 lessons 5 6 7 --- content/en/docs/tutorial-ai/Lesson6/_index.md | 2 -- content/en/docs/tutorial-ai/Lesson7/_index.md | 2 -- 2 files changed, 4 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson6/_index.md b/content/en/docs/tutorial-ai/Lesson6/_index.md index 8cd9af70..7887a7de 100644 --- a/content/en/docs/tutorial-ai/Lesson6/_index.md +++ b/content/en/docs/tutorial-ai/Lesson6/_index.md @@ -4,8 +4,6 @@ weight: 10 draft: true --- -# TO BE RERGANIZED AND REVIEWED! - > _Lesson 5_ introduced vision models and S3 storage. _Lesson 6_ introduces embeddings and vector databases. # Lesson 6 - Embeddings & Vector Databases diff --git a/content/en/docs/tutorial-ai/Lesson7/_index.md b/content/en/docs/tutorial-ai/Lesson7/_index.md index b479a30f..0aaa19e0 100644 --- a/content/en/docs/tutorial-ai/Lesson7/_index.md +++ b/content/en/docs/tutorial-ai/Lesson7/_index.md @@ -4,8 +4,6 @@ weight: 10 draft: true --- -# TO BE RERGANIZED AND REVIEWED! - > _Lesson 6_ introduced embeddings and vector search. _Lesson 7_ combines them with LLM generation to build a RAG system. # Lesson 7 - Retrieval Augmented Generation From d16944bf56639262e9d323709991f490c988e064 Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:38:35 +0200 Subject: [PATCH 09/10] fixed #92 published Tutorial MastroGPT --- content/en/docs/tutorial-ai/Lesson0/_index.md | 1 - content/en/docs/tutorial-ai/Lesson1/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson2/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson3/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson4/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson5/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson6/_index.md | 3 +-- content/en/docs/tutorial-ai/Lesson7/_index.md | 3 +-- content/en/docs/tutorial-ai/_index.md | 11 +++++------ 9 files changed, 12 insertions(+), 21 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson0/_index.md b/content/en/docs/tutorial-ai/Lesson0/_index.md index 9d9afb93..a70f445d 100644 --- a/content/en/docs/tutorial-ai/Lesson0/_index.md +++ b/content/en/docs/tutorial-ai/Lesson0/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 0 weight: 10 -draft: true --- # Welcome to the Course: Application Development with Apache Open Serverless diff --git a/content/en/docs/tutorial-ai/Lesson1/_index.md b/content/en/docs/tutorial-ai/Lesson1/_index.md index aa7ecdac..a6126b39 100644 --- a/content/en/docs/tutorial-ai/Lesson1/_index.md +++ b/content/en/docs/tutorial-ai/Lesson1/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 1 -weight: 10 -draft: true +weight: 20 --- diff --git a/content/en/docs/tutorial-ai/Lesson2/_index.md b/content/en/docs/tutorial-ai/Lesson2/_index.md index e321b138..1663659a 100644 --- a/content/en/docs/tutorial-ai/Lesson2/_index.md +++ b/content/en/docs/tutorial-ai/Lesson2/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 2 -weight: 10 -draft: true +weight: 30 --- > _Lesson 1_ introduced the first services and a simple chat action. _Lesson 2_ shows how to call an LLM directly and stream its answer through Open Serverless. diff --git a/content/en/docs/tutorial-ai/Lesson3/_index.md b/content/en/docs/tutorial-ai/Lesson3/_index.md index c5de7770..d26a40e8 100644 --- a/content/en/docs/tutorial-ai/Lesson3/_index.md +++ b/content/en/docs/tutorial-ai/Lesson3/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 3 -weight: 10 -draft: true +weight: 40 --- > _Lesson 2_ introduced LLM access, secrets, and streaming. _Lesson 3_ adds authenticated web actions, structured forms, and custom displays. diff --git a/content/en/docs/tutorial-ai/Lesson4/_index.md b/content/en/docs/tutorial-ai/Lesson4/_index.md index 281b501f..a7f25b70 100644 --- a/content/en/docs/tutorial-ai/Lesson4/_index.md +++ b/content/en/docs/tutorial-ai/Lesson4/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 4 -weight: 10 -draft: true +weight: 50 --- > _Lesson 3_ introduced authentication, forms, and custom displays. _Lesson 4_ builds a stateful assistant that remembers the conversation. diff --git a/content/en/docs/tutorial-ai/Lesson5/_index.md b/content/en/docs/tutorial-ai/Lesson5/_index.md index da275f4e..08a00d20 100644 --- a/content/en/docs/tutorial-ai/Lesson5/_index.md +++ b/content/en/docs/tutorial-ai/Lesson5/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 5 -weight: 10 -draft: true +weight: 60 --- # TO BE RERGANIZED AND REVIEWED! diff --git a/content/en/docs/tutorial-ai/Lesson6/_index.md b/content/en/docs/tutorial-ai/Lesson6/_index.md index 7887a7de..ecd1afb4 100644 --- a/content/en/docs/tutorial-ai/Lesson6/_index.md +++ b/content/en/docs/tutorial-ai/Lesson6/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 6 -weight: 10 -draft: true +weight: 70 --- > _Lesson 5_ introduced vision models and S3 storage. _Lesson 6_ introduces embeddings and vector databases. diff --git a/content/en/docs/tutorial-ai/Lesson7/_index.md b/content/en/docs/tutorial-ai/Lesson7/_index.md index 0aaa19e0..23b2ff33 100644 --- a/content/en/docs/tutorial-ai/Lesson7/_index.md +++ b/content/en/docs/tutorial-ai/Lesson7/_index.md @@ -1,7 +1,6 @@ --- title: Lesson 7 -weight: 10 -draft: true +weight: 80 --- > _Lesson 6_ introduced embeddings and vector search. _Lesson 7_ combines them with LLM generation to build a RAG system. diff --git a/content/en/docs/tutorial-ai/_index.md b/content/en/docs/tutorial-ai/_index.md index 078a1b6e..4884f955 100644 --- a/content/en/docs/tutorial-ai/_index.md +++ b/content/en/docs/tutorial-ai/_index.md @@ -1,13 +1,12 @@ --- -title: Tutorial -description: Showcase serverless development in action -weight: 10 -draft: true +title: Tutorial MastroGPT +description: Private AI tutorial with MastroGPT and Apache OpenServerless +weight: 60 --- -## AI Tutorial +## Tutorial MastroGPT We are transcribing all the lessons done for the MastroGPT AI tutorial. The ultimate goal is to build a consistent and well-structured documentation set based on the original course content, enabling easier access, referencing, and further processing. -The transcription process begins by leveraging the auto-generated captions provided by Google Drive for each recorded lesson. These initial drafts are then reviewed and refined on a per-lesson basis to ensure accuracy and clarity. \ No newline at end of file +The transcription process begins by leveraging the auto-generated captions provided by Google Drive for each recorded lesson. These initial drafts are then reviewed and refined on a per-lesson basis to ensure accuracy and clarity. From ed319d6e81f4b3f2e6f277bbd772b4302639b86e Mon Sep 17 00:00:00 2001 From: usualalteration Date: Wed, 17 Jun 2026 19:51:14 +0200 Subject: [PATCH 10/10] fixed #92 published Tutorial MastroGPT --- content/en/docs/tutorial-ai/Lesson5/_index.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/content/en/docs/tutorial-ai/Lesson5/_index.md b/content/en/docs/tutorial-ai/Lesson5/_index.md index 08a00d20..180dd24b 100644 --- a/content/en/docs/tutorial-ai/Lesson5/_index.md +++ b/content/en/docs/tutorial-ai/Lesson5/_index.md @@ -3,8 +3,6 @@ title: Lesson 5 weight: 60 --- -# TO BE RERGANIZED AND REVIEWED! - > _Lesson 4_ introduced stateful assistants with Redis. _Lesson 5_ adds computer vision and object storage. # Lesson 5 - Vision Models & S3 Storage