diff --git a/26.1.2/_downloads/110a2656d20dddee13a9d839f259851a/agentix.zip b/26.1.2/_downloads/110a2656d20dddee13a9d839f259851a/agentix.zip new file mode 100644 index 000000000..3cbdf0b2d Binary files /dev/null and b/26.1.2/_downloads/110a2656d20dddee13a9d839f259851a/agentix.zip differ diff --git a/26.1.2/_downloads/7661d03030174ba5165a854eea079407/constraints.txt b/26.1.2/_downloads/7661d03030174ba5165a854eea079407/constraints.txt new file mode 100644 index 000000000..e38f2225e --- /dev/null +++ b/26.1.2/_downloads/7661d03030174ba5165a854eea079407/constraints.txt @@ -0,0 +1,16 @@ +deprecated==1.2.18 +jinja2==3.1.6 +jq==1.8.0 +json_repair==0.30.0 +numpy==1.26.4 +pandas==2.2.3 +PyYAML==6.0.3 +pydantic==2.12.5 +httpx==0.28.1 +mcp==1.24.0 +pyagentspec==26.1.2 +opentelemetry-api==1.36.0 +opentelemetry-sdk==1.36.0 +litellm==1.84.0; python_version < "3.14" +fastapi==0.131.0 +anyio==4.11.0 diff --git a/26.1.2/_downloads/851f997ab457f57a59b6c42bfde5d608/usecase_prbot.ipynb b/26.1.2/_downloads/851f997ab457f57a59b6c42bfde5d608/usecase_prbot.ipynb new file mode 100644 index 000000000..984d92721 --- /dev/null +++ b/26.1.2/_downloads/851f997ab457f57a59b6c42bfde5d608/usecase_prbot.ipynb @@ -0,0 +1,1182 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "63042765-6b41-4a5a-a383-ae55141683f5", + "metadata": {}, + "source": [ + "# Build a Simple Code Review Assistant" + ] + }, + { + "cell_type": "markdown", + "id": "0836dd53-861d-423e-9eec-ce6b44091827", + "metadata": {}, + "source": [ + "> **Prerequisites**\n", + "> \n", + "> This guide does not assume any prior knowledge about Project WayFlow. However, it assumes the reader has a basic knowledge of LLMs.\n", + ">\n", + "> You will need a working installation of WayFlow - see [Installation](https://TODO/development/docs/installation.html)." + ] + }, + { + "cell_type": "markdown", + "id": "7a3048f2-16ac-4f27-aea0-80ddc22bc32f", + "metadata": {}, + "source": [ + "## Learning goals\n", + "\n", + "In this use-case tutorial, you will build a more advanced WayFlow application, a **Pull Request (PR) Reviewing Assistant**, using a WayFlow Flow to automate basic reviews of Python source code.\n", + "\n", + "In this tutorial you will:\n", + "\n", + "1. Learn the basics of using Flows to build an assistant.\n", + "2. Learn how to compose multiple sub-flows to create a more complex Flow.\n", + "3. Learn more about building Tools that can be used within your Flows.\n", + "\n", + "## Introduction to the task\n", + "\n", + "Code reviews are crucial for maintaining code quality and reviewers often spend considerable time pointing out routine issues such as the presence of debug statements, formatting inconsistencies, or common coding convention violations that may not be fully captured by static code analysis tools. This consumes valuable time that could be spent on reviewing more important things such as the core logic, architecture, and business requirements.\n", + "\n", + "Building an agent with WayFlow to perform such code reviews has a number of advantages:\n", + "\n", + "1. Review rules can be written using natural language. This can make an agent much more flexible than simple static checker.\n", + "2. More general issues can be captured. You can allow the LLM to infer from the rule to more general cases that could be missed by a simple static checker.\n", + "3. New review rules can be learned by looking at comment traces\n", + "\n", + "In this tutorial, you will create a WayFlow Flow assistant designed to scan Python pull requests for common oversights such as:\n", + "\n", + "1. Having TODO comments without associated tickets.\n", + "2. Using unclear or ambiguous variable naming.\n", + "3. Using risky Python code practices such as mutable defaults.\n", + "\n", + "To build this assistant you will break the task into configuration and two sub-flows that will be composed into a single flow:" + ] + }, + { + "cell_type": "markdown", + "id": "8b9034e4-9b46-4730-9bd8-d9b142906183", + "metadata": {}, + "source": [ + "![PR Review Bot Chematic Diagram.](https://TODO/development/docs/_images/prbot_main.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "93d47038-ef53-4ef4-a921-56edd2e40007", + "metadata": {}, + "source": [ + "\n", + "1. Configure your application, choose an LLM and import required modules [Part 1].\n", + "2. The first sub-flow retrieves and diffs information from a local codebase in a Git repository [*Part 2*].\n", + "3. The second sub-flow iterates over the file diffs using a [MapStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep) and generates comments with an LLM using the [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep) [Step 3].\n", + "\n", + "You will also learn how to extract information using the [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) and the [ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep), and how to build and execute tools with the [ServerTool](https://TODO/development/docs/api/tools.html#wayflowcore.tools.servertools.ServerTool) and the [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep)." + ] + }, + { + "cell_type": "markdown", + "id": "3c69bdc8-5944-4141-956f-1fdb7a02fe3e", + "metadata": {}, + "source": [ + "> **Note**\n", + ">\n", + "> This is not a production-ready code review assistant that can be used as-is." + ] + }, + { + "cell_type": "markdown", + "id": "ec6ba701-440f-4978-846f-a17155b9bd97", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "First, let’s set up the environment. For this tutorial you need to have wayflowcore installed.\n", + "\n", + "Next download the example codebase Git repository, :download:`example codebase Git repository <../_static/usecases/agentix.zip>`. This will be used\n", + "\n", + "Extract the codebase Git repository folder from the compressed archive. Make a note of where the codebase Git repository is extracted to.\n", + "\n", + "\n", + "## Part 1: Imports and LLM configuration\n", + "\n", + "First, set up the environment. For this tutorial you need to have wayflowcore installed, for additional information, read the [Installation guide](https://TODO/development/docs/installation.html).Then configure a Large Language Model (LLM) to use. WayFlow supports several LLMs API providers, to learn more about the supported LLM providers please read our guide, how to use LLMs from different providers.\n", + "\n", + "Choose an LLM from one of the options below:" + ] + }, + { + "cell_type": "markdown", + "id": "4c85391f", + "metadata": {}, + "source": [ + "> **Note**\n", + "> API keys should never be stored in code. Use environment variables and/or tools such as [python-dotenv](https://pypi.org/project/python-dotenv) instead.\n", + ">" + ] + }, + { + "cell_type": "markdown", + "id": "3861a773-7e0d-4690-82e3-e44fff6cc234", + "metadata": {}, + "source": [ + "#### OCI GenAI (Cohere)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f468b58e-4c32-4e60-b544-d3aac70a0eae", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.models import OCIGenAIModel\n", + "\n", + "if __name__ == \"__main__\":\n", + "\n", + " llm = OCIGenAIModel(\n", + " model_id=\"cohere.model-id\",\n", + " service_endpoint=\"https://url-to-service-endpoint.com\",\n", + " compartment_id=\"compartment-id\",\n", + " auth_type=\"API_KEY\",\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "5c994912-8a3d-4ca7-918f-363646ba8d4f", + "metadata": {}, + "source": [ + "#### Hosted Model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9aa848f-ba08-4dd9-a5dd-881cf40d57ac", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.models import VllmModel\n", + "\n", + "llm = VllmModel(\n", + " model_id=\"model-id\",\n", + " host_port=\"VLLM_HOST_PORT\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f984143e-bde7-45f9-ab5f-0de37ebb2f79", + "metadata": {}, + "source": [ + "#### Ollama" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2cced4a6-7528-46d2-bb7e-483b4d0c856e", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.models import OllamaModel\n", + "\n", + "llm = OllamaModel(\n", + " model_id=\"model-id\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "383dcb91", + "metadata": {}, + "source": [ + "Be cautious when using external LLM providers and ensure that you comply with your organization's security policies and any applicable laws and regulations. Consider using a self-hosted LLM solution or a provider that offers on-premises deployment options if you need to maintain strict control over your code and data." + ] + }, + { + "cell_type": "markdown", + "id": "0d29f1ad", + "metadata": {}, + "source": [ + "## Part 2: Retrieve the PR diff information\n", + "\n", + "The first phase of the assistant requires retrieving information about the code diffs from a code repository. You have already extracted the sample\n", + "codebase git repository to your local environment.\n", + "\n", + "This will be a sub-flow that consists of two simple steps:\n", + "\n", + "* [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep): that collects PR diff information using a Python subprocess to run the git command.\n", + "* [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep): which separate the raw diff information into diffs for each file.\n", + "\n", + "First, take a look at what a diff looks like. The following example shows how a real diff appears when using Git:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "384c5066", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "MOCK_DIFF = \"\"\"\n", + "diff --git src://calculators/utils.py dst://calculators/utils.py\n", + "index 12345678..90123456 100644\n", + "--- src://calculators/utils.py\n", + "+++ dst://calculators/utils.py\n", + "@@ -10,6 +10,15 @@\n", + "\n", + " def calculate_total(data):\n", + " # TODO: implement tax calculation\n", + " return data\n", + "\n", + "+def get_items(items=[]):\n", + "+ result = []\n", + "+ for item in items:\n", + "+ result.append(item * 2)\n", + "+ return result\n", + "+\n", + "+def process_numbers(numbers):\n", + "+ res = []\n", + "+ for x in numbers:\n", + "+ res.append(x + 1)\n", + "+ return res\n", + "+\n", + " def calculate_average(numbers):\n", + " return sum(numbers) / len(numbers)\n", + "\n", + "\n", + "diff --git src://example/utils.py dst://example/utils.py\n", + "index 000000000..123456789\n", + "--- /dev/null\n", + "+++ dst://example/utils.py\n", + "@@ -0,0 +1,20 @@\n", + "+# Copyright (C) 2024 Oracle and/or its affiliates.\n", + "+\n", + "+def calculate_sum(numbers=[]):\n", + "+ total = 0\n", + "+ for num in numbers:\n", + "+ total += num\n", + "+ return total\n", + "+\n", + "+\n", + "+def process_data(data):\n", + "+ # TODO: Handle exceptions here\n", + "+ result = data * 2\n", + "+ return result\n", + "+\n", + "+\n", + "+def main():\n", + "+ numbers = [1, 2, 3, 4, 5]\n", + "+ result = calculate_sum(numbers)\n", + "+ print(\"Sum:\", result)\n", + "+ data = 10\n", + "+ processed_data = process_data(data)\n", + "+ print(\"Processed Data:\", processed_data)\n", + "+\n", + "+\n", + "+if __name__ == \"__main__\":\n", + "+ main()\n", + "\"\"\".strip()" + ] + }, + { + "cell_type": "markdown", + "id": "f674ce1d", + "metadata": {}, + "source": [ + "**Reading a diff**: Removals are identified by the \"-\" marks and additions by the \"+\" marks. In this example, there were only additions.\n", + "\n", + "The diff above contains information about two files, ``calculators/utils.py`` and ``example/utils.py``. This is an example diff and it is different\n", + "from the diff that will be generated from the sample codebase. It is included here to show how a Git diff looks and is shorter than the diff that you generate from the sample codebase." + ] + }, + { + "cell_type": "markdown", + "id": "0fd860ac", + "metadata": {}, + "source": [ + "### Build a tool\n", + "\n", + "To extract the diffs from the codebase git repository you need to create a tool, using the [ServerTool](https://TODO/development/docs/api/tools.html#wayflowcore.tools.servertools.ServerTool), that will do this.\n", + "\n", + "The function, ``local_get_pr_diff_tool``, in the code below does the work of extracting the diffs by running the `git diff HEAD` shell command and capturing the output. It uses a subprocess to run the shell command.\n", + "\n", + "To turn this function into a WayFlow tool, a `@tool` annotation is used to create a [ServerTool](https://TODO/development/docs/api/tools.html#wayflowcore.tools.servertools.ServerTool) from the function." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "61e2851c-afe4-4583-9985-442d3c29dec0", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.tools import tool\n", + "\n", + "@tool(description_mode=\"only_docstring\")\n", + "def local_get_pr_diff_tool(repo_dirpath: str) -> str:\n", + " \"\"\"\n", + " Retrieves code diff with a git command given the\n", + " path to the repository root folder.\n", + " \"\"\"\n", + " import subprocess\n", + "\n", + " result = subprocess.run(\n", + " [\"git\", \"diff\", \"HEAD\"],\n", + " capture_output=True,\n", + " cwd=repo_dirpath,\n", + " text=True,\n", + " )\n", + " return result.stdout.strip()" + ] + }, + { + "cell_type": "markdown", + "id": "397299c9-b44e-4879-965a-455286fc4503", + "metadata": {}, + "source": [ + "### Building the steps and the sub-flow\n", + "\n", + "Let's write the code for the first sub-flow." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9647e967-908e-4fa9-bb04-1f998b92fabd", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.controlconnection import ControlFlowEdge\n", + "from wayflowcore.dataconnection import DataFlowEdge\n", + "from wayflowcore.flow import Flow\n", + "from wayflowcore.property import StringProperty\n", + "from wayflowcore.steps import RegexExtractionStep, StartStep, ToolExecutionStep\n", + "\n", + "# Step Names\n", + "START_STEP = \"start_step\"\n", + "GET_PR_DIFF_STEP = \"get_pr_diff\"\n", + "EXTRACT_INTO_LIST_OF_FILE_DIFF_STEP = \"extract_into_list_of_file_diff\"\n", + "\n", + "# IO Variable Names\n", + "REPO_DIRPATH_IO = \"$repo_dirpath_io\"\n", + "PR_DIFF_IO = \"$raw_pr_diff\"\n", + "FILE_DIFF_LIST_IO = \"$file_diff_list\"\n", + "\n", + "# Define the steps\n", + "\n", + "start_step = StartStep(input_descriptors=[StringProperty(name=REPO_DIRPATH_IO)])\n", + "\n", + "# Step 1: Retrieve the pull request diff using the local tool\n", + "get_pr_diff_step = ToolExecutionStep(\n", + " tool=local_get_pr_diff_tool,\n", + " raise_exceptions=True,\n", + " input_mapping={\"repo_dirpath\": REPO_DIRPATH_IO},\n", + " output_mapping={ToolExecutionStep.TOOL_OUTPUT: PR_DIFF_IO},\n", + ")\n", + "\n", + "# Step 2: Extract the file diffs from the raw diff using a regular expression\n", + "extract_into_list_of_file_diff_step = RegexExtractionStep(\n", + " regex_pattern=\"(diff --git[\\s\\S]*?)(?=diff --git|$)\",\n", + " return_first_match_only=False,\n", + " input_mapping={RegexExtractionStep.TEXT: PR_DIFF_IO},\n", + " output_mapping={RegexExtractionStep.OUTPUT: FILE_DIFF_LIST_IO},\n", + ")\n", + "\n", + "# Define the sub flow\n", + "retrieve_diff_subflow = Flow(\n", + " begin_step=start_step,\n", + " steps={\n", + " START_STEP: start_step,\n", + " GET_PR_DIFF_STEP: get_pr_diff_step,\n", + " EXTRACT_INTO_LIST_OF_FILE_DIFF_STEP: extract_into_list_of_file_diff_step,\n", + " },\n", + " control_flow_edges=[\n", + " ControlFlowEdge(source_step=start_step, destination_step=get_pr_diff_step),\n", + " ControlFlowEdge(\n", + " source_step=get_pr_diff_step, destination_step=extract_into_list_of_file_diff_step\n", + " ),\n", + " ControlFlowEdge(source_step=extract_into_list_of_file_diff_step, destination_step=None),\n", + " ],\n", + " data_flow_edges=[\n", + " DataFlowEdge(\n", + " source_step=start_step,\n", + " source_output=REPO_DIRPATH_IO,\n", + " destination_step=get_pr_diff_step,\n", + " destination_input=REPO_DIRPATH_IO,\n", + " ),\n", + " DataFlowEdge(\n", + " source_step=get_pr_diff_step,\n", + " source_output=PR_DIFF_IO,\n", + " destination_step=extract_into_list_of_file_diff_step,\n", + " destination_input=PR_DIFF_IO,\n", + " ),\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "ffaefeb2-f162-43af-ac69-fec7b5314825", + "metadata": {}, + "source": [ + "**API Reference:** [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow) | [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) | [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep)\n", + "\n", + "The code does the following:\n", + "\n", + "#. It lists the names of the steps and input/output variables for the sub-flow.\n", + "#. It then creates the different steps within the sub-flow.\n", + "#. Finally, it instantiates the sub-flow. This will be covered in more detail later in the tutorial.\n", + "\n", + "For clarity, the variable names are also prefixed with a dollar ($) sign. This is not necessary and is only done for code clarity. The variable\n", + "`REPO_DIRPATH_IO` is used to hold the file path to the sample codebase Git repository and you will use this to pass in the location of the codebase Git repository.\n", + "\n", + "Additionally, you can give explicit names to the input/output variables used in the Flow, e.g. \"$repo_dirpath_io\" for the variable holding the path to the local repository. Finally, we define those explicit names as string variables (e.g. ``REPO_DIRPATH_IO``) to minimize the number of magic strings in the code.\n", + "\n", + "> **See also**\n", + "> To learn about the basics of Flows, check out our, :doc:`introductory tutorial on WayFlow Flows `.\n", + "\n", + "Now take a look at each of the steps used in the sub-flow in more detail." + ] + }, + { + "cell_type": "markdown", + "id": "3f371a99-9dda-491f-a4a7-b588081ca528", + "metadata": {}, + "source": [ + "#### Get the PR diff, `get_pr_diff_step`\n", + "\n", + "This uses a [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep) to gather the diff information - see the notes on how this is done earlier. When creating it, you need to\n", + "provide the following:\n", + "\n", + "* `tool`: Specifies the tool that will called within the step. This is the tool that was created earlier, `local_get_pr_diff_tool`.\n", + "* `raise_exceptions`: Whether to raise exceptions generated by the tool that is called. Here it is set to `True` and so exceptions will be raised.\n", + "* `input_mapping`: Specifies the names used for the input parameters of the step. See :ref:`ToolExecutionStep ` for more details on using an ``input_mapping`` with this type of step.\n", + "* `output_mapping`: Specifies the name used foe the output parameter of the step. The name held in ``PR_DIFF_IO`` will be mapped to the name for the output parameter of the step. Again, see :ref:`ToolExecutionStep ` for more details on using an ``output_mapping`` with this type of step." + ] + }, + { + "cell_type": "markdown", + "id": "240c6302-9bac-4d52-90ea-130327b86baa", + "metadata": {}, + "source": [ + "#### Extract file diffs into a list, `extract_into_list_of_file_diff_step`\n", + "\n", + "You now have the diff information from the PR. This step performs a regex extraction on the raw diff text to extract the code to review.\n", + "\n", + "Use a [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) to perform this action. When creating the step, you need to provide the following:\n", + "\n", + "* `regex_pattern`: The regex pattern for the extraction. This uses `re.findall` underneath.\n", + "* `return_first_match_only`: You want to return all results, so set this to `False`.\n", + "* `input_mapping`: Specifies the names used for the input parameters of the step. The input parameter will be mapped to the name, held in ``PR_DIFF_IO``. See :ref:`RegexExtractionStep ` for more details on using an ``input_mapping`` with this type of step.\n", + "* `output_mapping`: Specifies the name used for the output parameter of the step. Here, the default name ``RegexExtractionStep.TEXT`` is renamed to the name defined in ``PR_DIFF_IO``. Again, see :ref:`RegexExtractionStep ` for more details on using an ``output_mapping`` with this type of step.\n", + "\n", + "**About the pattern:**\n", + "\n", + " (diff --git[\\s\\S]*?)(?=diff --git|$)\n", + "\n", + "The pattern looks for text starting with `diff --git`, followed by any characters (both whitespace `[\\s]` and non-whitespace `[\\S]`), until it\n", + "encounters either another `diff --git` or the end of the text ($). However, it does not include the next `diff --git` or the end in the match.\n", + "\n", + "The `\\*?` makes it \"lazy\" or non-greedy, meaning it takes the shortest possible match, rather than the longest.\n", + "\n", + "> **Tip**\n", + "> Recent Large Language Models are very helpful tools to create, debug and explain Regex patterns given a natural language\n", + "> description.\n", + "\n", + "Finally, create the sub-flow using the [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow) class. You specify the steps in the Flow, the starting step of the Flow, the transitions between steps and how data, from the variables, is to pass from one step to the next." + ] + }, + { + "cell_type": "markdown", + "id": "bbd0b451-d607-46a0-95ef-ffe5ac2b0087", + "metadata": {}, + "source": [ + "### Defining a Flow\n", + "\n", + "Defining the Flow is the last step in the code shown above. There are a couple of things that are worth highlighting:\n", + "\n", + "* `begin_step`: A start step needs to be defined for a [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow).\n", + "* `steps`: A list of the steps that make up the [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow).\n", + "* `control_flow_edges`: The transitions between the steps in the [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow) are defined as [ControlFlowEdges](https://TODO/development/docs/api/flows.html#wayflowcore.controlconnection.ControlFlowEdge). They have a `source_step`, which defines the start of a transition, and a `destination_step`, which defines the destination of a transition. All transitions for the flow will need to be defined.\n", + "* `data_flow_edges`: Maps the variables between steps connected by a transition using [DataFlowEdges](https://TODO/development/docs/api/flows.html#wayflowcore.dataconnection.DataFlowEdge). It maps variables from a source step into variables in a destination step. You only need to do this for the variables that need to be passed between steps." + ] + }, + { + "cell_type": "markdown", + "id": "d3530be6-bd15-45bb-a095-60478566f484", + "metadata": {}, + "source": [ + "### Testing the flow\n", + "\n", + "You can test this sub-flow by creating an assistant conversation with the `start_conversation` method and specifying the inputs, in this case\n", + "the location of the Git repository. The conversation can then be executed with the conversation `execute` method, into which you pass the conversation object.\n", + "This returns an object that represents the status of the conversation which you can check to confirm that the conversation has successfully finished.\n", + "\n", + "The code below shows how the inputs are passed in. Set the `PATH_TO_DIR` to the actual path you extracted the sample codebase Git repository to. You then extract the outputs from the conversation.\n", + "\n", + "Create a variable that will point to the location of your actual codebase sample Git repository." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "25f14f23-56e0-4a49-87ad-6d960a97081d", + "metadata": {}, + "outputs": [], + "source": [ + "# Replace the path below with the path to your actual codebase sample git repository.\n", + "#PATH_TO_DIR = \"path/to/repository_root\"\n", + "PATH_TO_DIR = \"/Users/krifoste/Downloads/agentix\"" + ] + }, + { + "cell_type": "markdown", + "id": "48243207-a001-42a0-b21e-fc60f0be85e8", + "metadata": {}, + "source": [ + "The full code for testing the sub-flow is shown below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6ca6a9e-41fe-47e3-823f-5f473e54c8f2", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.executors.executionstatus import FinishedStatus\n", + "\n", + "test_conversation = retrieve_diff_subflow.start_conversation(\n", + " inputs={\n", + " REPO_DIRPATH_IO: PATH_TO_DIR,\n", + " }\n", + ")\n", + "\n", + "execution_status = test_conversation.execute()\n", + "\n", + "assert isinstance(execution_status, FinishedStatus)\n", + "FILE_DIFF_LIST = execution_status.output_values[FILE_DIFF_LIST_IO]\n", + "\n", + "print(FILE_DIFF_LIST[0])" + ] + }, + { + "cell_type": "markdown", + "id": "e9e6efb8-15e6-44a3-9b5a-1259de2489c7", + "metadata": {}, + "source": [ + "**API Reference:** [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow)" + ] + }, + { + "cell_type": "markdown", + "id": "55cd1138-11d8-419d-9306-97dae8dee46b", + "metadata": {}, + "source": [ + "## Part 3: Review the list of diffs\n", + "\n", + "Now that we have a list of diffs for each file, we can review them and generate comments using an LLM.\n", + "\n", + "This task can be broken into a sub-flow made up of five steps:\n", + "\n", + "* [OutputMessageStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.outputmessagestep.OutputMessageStep): This converts the file diff list into a string to be processed by the following steps.\n", + "* [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep): This prefixes the diffs with line numbers for additional context to the LLM.\n", + "* [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep): This extracts the file path from the diff string.\n", + "* [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep): This generates comments using the LLM based on a list of user-defined checks.\n", + "* [ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep): This extracts the comments and lines they apply to from the LLM output." + ] + }, + { + "cell_type": "markdown", + "id": "ac32ff65-f848-41fd-858f-875df8965e12", + "metadata": {}, + "source": [ + "![PR Review Bot Chematic Diagram.](https://TODO/development/docs/_images/prbot_generate_comment.svg)" + ] + }, + { + "cell_type": "markdown", + "id": "dd8f8478-e71f-48d2-999d-123920167faa", + "metadata": {}, + "source": [ + "### Build the tools and checks\n", + "\n", + "Before creating the steps and sub-flow to generate the comments, it is important to define the list of checks the assistant should perform,\n", + "along with any specific instructions. Additionally, a tool must be created to prefix the diffs with line numbers, allowing the LLM to determine\n", + "where to add comments.\n", + "\n", + "Below is the full code to achieve this. It is broken into sections so that you can see, in detail, what is happening in each part." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0bb2b2d3-861c-4a01-8c68-b687df26014d", + "metadata": {}, + "outputs": [], + "source": [ + "PR_BOT_CHECKS = [\n", + " \"\"\"\n", + "Name: TODO_WITHOUT_TICKET\n", + "Description: TODO comments should reference a ticket number for tracking.\n", + "Example code:\n", + "```python\n", + "# TODO: Add validation here\n", + "def process_user_input(data):\n", + " return data\n", + "```\n", + "Example comment:\n", + "[BOT] TODO_WITHOUT_TICKET: TODO comment should reference a ticket number for tracking (e.g., \"TODO: Add validation here (TICKET-1234)\").\n", + "\"\"\",\n", + " \"\"\"\n", + "Name: MUTABLE_DEFAULT_ARGUMENT\n", + "Description: Using mutable objects as default arguments can lead to unexpected behavior.\n", + "Example code:\n", + "```python\n", + "def add_item(item, items=[]):\n", + " items.append(item)\n", + " return items\n", + "```\n", + "Example comment:\n", + "[BOT] MUTABLE_DEFAULT_ARGUMENT: Avoid using mutable default arguments. Use None and initialize in the function: `def add_item(item, items=None): items = items or []`\n", + "\"\"\",\n", + " \"\"\"\n", + "Name: NON_DESCRIPTIVE_NAME\n", + "Description: Variable names should clearly indicate their purpose or content.\n", + "Example code:\n", + "```python\n", + "def process(lst):\n", + " res = []\n", + " for i in lst:\n", + " res.append(i * 2)\n", + " return res\n", + "```\n", + "Example comment:\n", + "[BOT] NON_DESCRIPTIVE_NAME: Use more descriptive names: 'lst' could be 'numbers', 'res' could be 'doubled_numbers', 'i' could be 'number'\n", + "\"\"\",\n", + "]\n", + "\n", + "CONCATENATED_CHECKS = \"\\n\\n---\\n\\n\".join(check for check in PR_BOT_CHECKS)\n", + "\n", + "PROMPT_TEMPLATE = \"\"\"You are a very experienced code reviewer. You are given a git diff on a file: {{filename}}\n", + "\n", + "## Context\n", + "The git diff contains all changes of a single file. All lines are prepended with their number. Lines without line number where removed from the file.\n", + "After the line number, a line that was changed has a \"+\" before the code. All lines without a \"+\" are just here for context, you will not comment on them.\n", + "\n", + "## Input\n", + "### Code diff\n", + "{{diff}}\n", + "\n", + "## Task\n", + "Your task is to review these changes, according to different rules. Only comment lines that were added, so the lines that have a + just after the line number.\n", + "The rules are the following:\n", + "\n", + "{{checks}}\n", + "\n", + "### Response Format\n", + "You need to return a review as a json as follows:\n", + "```json\n", + "[\n", + " {\n", + " \"content\": \"the comment as a text\",\n", + " \"suggestion\": \"if the change you propose is a single line, then put here the single line rewritten that includes your proposal change. IMPORTANT: a single line, which will erase the current line. Put empty string if no suggestion of if the suggestion is more than a single line\",\n", + " \"line\": \"line number where the comment applies\"\n", + " },\n", + " …\n", + "]\n", + "```\n", + "Please use triple backticks ``` to delimitate your JSON list of comments. Don't output more than 5 comments, only comment the most relevant sections.\n", + "If there are no comments and the code seems fine, just output an empty JSON list.\"\"\"\n", + "\n", + "\n", + "# Tools\n", + "@tool(description_mode=\"only_docstring\")\n", + "def format_git_diff(diff_text: str) -> str:\n", + " \"\"\"\n", + " Formats a git diff with line numbers everywhere.\n", + " \"\"\"\n", + "\n", + " def pad_number(number: int, width: int) -> str:\n", + " \"\"\"Right-align a number with specified width using space padding.\"\"\"\n", + " return str(number).rjust(width)\n", + "\n", + " LINE_NUMBER_WIDTH = 5\n", + " PADDING_WIDTH = LINE_NUMBER_WIDTH + 1\n", + " current_line_number = 0\n", + " formatted_lines = []\n", + "\n", + " for line in diff_text.split(\"\\n\"):\n", + " # Handle diff header lines (e.g., \"@@ -1,7 +1,6 @@\")\n", + " if line.startswith(\"@@\"):\n", + " try:\n", + " # Extract the starting line number and line count\n", + " _, position_info, _ = line.split(\"@@\")\n", + " new_file_info = position_info.split()[1][1:] # Remove the '+' prefix\n", + " start_line, line_count = map(int, new_file_info.split(\",\"))\n", + "\n", + " current_line_number = start_line\n", + " formatted_lines.append(line)\n", + " continue\n", + "\n", + " except (ValueError, IndexError):\n", + " raise ValueError(f\"Invalid diff header format: {line}\")\n", + "\n", + " # Handle content lines\n", + " if current_line_number > 0 and line:\n", + " if not line.startswith(\"-\"):\n", + " # Add line number for added/context lines\n", + " line_prefix = pad_number(current_line_number, LINE_NUMBER_WIDTH)\n", + " formatted_lines.append(f\"{line_prefix} {line}\")\n", + " current_line_number += 1\n", + " else:\n", + " # Just add padding for removal lines\n", + " formatted_lines.append(\" \" * PADDING_WIDTH + line)\n", + "\n", + " return \"\\n\".join(formatted_lines)" + ] + }, + { + "cell_type": "markdown", + "id": "5ed45264-e1cd-41fd-b6ce-71e832801c5c", + "metadata": {}, + "source": [ + "**API Reference:** [ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep) | [MapStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep) |\n", + "[OutputMessageStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.outputmessagestep.OutputMessageStep) | [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep) | [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep)" + ] + }, + { + "cell_type": "markdown", + "id": "ce5fdb60-f853-4515-8423-26393064784f", + "metadata": {}, + "source": [ + "#### Checks and LLM instructions\n", + "\n", + "You will use three simple checks that are shown below. For each check you specify a name, a description of what the LLM should be checking,\n", + "as well as a code and expected comment example so that the LLM gets a better understanding of what the task is about.\n", + "\n", + "The prompt uses a simple structure:\n", + "\n", + "#. **Role Definition**: Define who/what you want the LLM to act as (e.g., \"You are a very experienced code reviewer\").\n", + "#. **Context Section**: Provide relevant background information or specific circumstances that frame the task.\n", + "#. **Input Section**: Specify the exact information, data, or materials that the LLM will be provided with.\n", + "#. **Task Section**: Clearly state what you want the LLM to do with the input provided.\n", + "#. **Response Format Section**: Define how you want the response to be structured or formatted (e.g., bullet points, JSON, with XML tags, and so on).\n", + "\n", + "The prompts are defined in the array, `PR_BOT_CHECKS`. The individual prompts for the checks are then concatenated into a single string,\n", + "`CONCATENATED_CHECKS`, so that it can be used inside the system prompt you will be passing to the LLM.\n", + "\n", + "Define a system prompt, or prompt template, `PROMPT_TEMPLATE`. It contains placeholders for the diff and the checks that will be replaced when specialising\n", + "the prompt for each diff.\n", + "\n", + "> **Tip**\n", + "> **How to write high-quality prompts**\n", + ">\n", + "> There is no consensus on what makes the best LLM prompt. However, it is noted that for recent LLMs, a great strategy\n", + "> to use to prompt an LLM is simply to be very specific about the task to be solved, giving enough context and explaining\n", + "> potential edge cases to consider.\n", + ">\n", + "> Given a prompt, try to determine whether giving the set of instructions to an experienced colleague, that has no prior\n", + "> context about the task, to solve would be sufficient for them to get to the intended result.\n", + "\n", + "#### Diff formatting tool\n", + "\n", + "You next need to create a tool to format the diffs in a manner that makes them consumable\n", + "by the LLM. A tool, as you will have already seen, is a simple wrapper around a `python` callable that makes it useable within a flow.\n", + "\n", + "The function, `format_git_diff`, in the code above does the work of formatting the diffs.\n", + "\n", + "> **See also**\n", + "> \n", + "> For more information about WayFlow tools please read our guide, [How to use tools](https://TODO/development/docs/howtoguides/howto_build_assistants_with_tools.html).\n", + ">\n", + "\n", + "#### Building the steps and the sub-flow\n", + "\n", + "With the prompts and diff formatting tool written you can now build the second sub-flow. This sub-flow will iterate over the diffs, generated\n", + "previously, and then use an LLM to generate review comments from them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6dd8b04-4682-417b-8e9a-08fb47cd16bb", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore._utils._templating_helpers import render_template_partially\n", + "from wayflowcore.property import AnyProperty, DictProperty, ListProperty, StringProperty\n", + "from wayflowcore.steps import (\n", + " ExtractValueFromJsonStep,\n", + " MapStep,\n", + " OutputMessageStep,\n", + " PromptExecutionStep,\n", + " ToolExecutionStep,\n", + ")\n", + "\n", + "# Step Names\n", + "FORMAT_DIFF_TO_STRING_STEP = \"format_diff_to_string\"\n", + "EXTRACT_FILE_PATH_STEP = \"extract_file_path\"\n", + "EXTRACT_COMMENTS_FROM_JSON_STEP = \"extract_comments_from_json\"\n", + "GENERATE_COMMENTS_STEP = \"generate_comments\"\n", + "ADD_LINES_ON_DIFF_STEP = \"add_lines_on_diff\"\n", + "\n", + "# IO Variable Names\n", + "DIFF_TO_STRING_IO = \"$diff_to_string\"\n", + "DIFF_WITH_LINES_IO = \"$diff_with_lines\"\n", + "FILEPATH_IO = \"$filename\"\n", + "JSON_COMMENTS_IO = \"$json_comments\"\n", + "EXTRACTED_COMMENTS_IO = \"$extracted_comments\"\n", + "NESTED_COMMENT_LIST_IO = \"$nested_comment_list\"\n", + "FILEPATH_LIST_IO = \"$filepath_list\"\n", + "\n", + "# Define the steps\n", + "\n", + "# Step 1: Format the diff to a string\n", + "format_diff_to_string_step = OutputMessageStep(\n", + " message_template=\"{{ message | string }}\",\n", + " output_mapping={OutputMessageStep.OUTPUT: DIFF_TO_STRING_IO},\n", + ")\n", + "\n", + "# Step 2: Add lines on the diff using a tool\n", + "add_lines_on_diff_step = ToolExecutionStep(\n", + " tool=format_git_diff,\n", + " input_mapping={\"diff_text\": DIFF_TO_STRING_IO},\n", + " output_mapping={ToolExecutionStep.TOOL_OUTPUT: DIFF_WITH_LINES_IO},\n", + ")\n", + "\n", + "# Step 3: Extract the file path from the diff string using a regular expression\n", + "extract_file_path_step = RegexExtractionStep(\n", + " regex_pattern=r\"diff --git a/(.+?) b/\",\n", + " return_first_match_only=True,\n", + " input_mapping={RegexExtractionStep.TEXT: DIFF_TO_STRING_IO},\n", + " output_mapping={RegexExtractionStep.OUTPUT: FILEPATH_IO},\n", + ")\n", + "\n", + "# Step 4: Generate comments using a prompt\n", + "generate_comments_step = PromptExecutionStep(\n", + " prompt_template=render_template_partially(PROMPT_TEMPLATE, {\"checks\": CONCATENATED_CHECKS}),\n", + " llm=llm,\n", + " input_mapping={\"diff\": DIFF_WITH_LINES_IO, \"filename\": FILEPATH_IO},\n", + " output_mapping={PromptExecutionStep.OUTPUT: JSON_COMMENTS_IO},\n", + ")\n", + "\n", + "# Step 5: Extract comments from the JSON output\n", + "# Define the value type for extracted comments\n", + "comments_valuetype = ListProperty(\n", + " name=\"values\",\n", + " description=\"The extracted comments content and line number\",\n", + " item_type=DictProperty(value_type=AnyProperty()),\n", + ")\n", + "extract_comments_from_json_step = ExtractValueFromJsonStep(\n", + " output_values={comments_valuetype: '[.[] | {\"content\": .[\"content\"], \"line\": .[\"line\"]}]'},\n", + " retry=True,\n", + " llm=llm,\n", + " input_mapping={ExtractValueFromJsonStep.TEXT: JSON_COMMENTS_IO},\n", + " output_mapping={\"values\": EXTRACTED_COMMENTS_IO},\n", + ")\n", + "\n", + "# Define the sub flow to generate comments for each file diff\n", + "generate_comments_subflow = Flow(\n", + " begin_step=format_diff_to_string_step,\n", + " steps={\n", + " FORMAT_DIFF_TO_STRING_STEP: format_diff_to_string_step,\n", + " ADD_LINES_ON_DIFF_STEP: add_lines_on_diff_step,\n", + " EXTRACT_FILE_PATH_STEP: extract_file_path_step,\n", + " GENERATE_COMMENTS_STEP: generate_comments_step,\n", + " EXTRACT_COMMENTS_FROM_JSON_STEP: extract_comments_from_json_step,\n", + " },\n", + " control_flow_edges=[\n", + " ControlFlowEdge(format_diff_to_string_step, add_lines_on_diff_step),\n", + " ControlFlowEdge(add_lines_on_diff_step, extract_file_path_step),\n", + " ControlFlowEdge(extract_file_path_step, generate_comments_step),\n", + " ControlFlowEdge(generate_comments_step, extract_comments_from_json_step),\n", + " ControlFlowEdge(extract_comments_from_json_step, None),\n", + " ],\n", + " data_flow_edges=[\n", + " DataFlowEdge(\n", + " format_diff_to_string_step, DIFF_TO_STRING_IO, add_lines_on_diff_step, DIFF_TO_STRING_IO\n", + " ),\n", + " DataFlowEdge(\n", + " format_diff_to_string_step, DIFF_TO_STRING_IO, extract_file_path_step, DIFF_TO_STRING_IO\n", + " ),\n", + " DataFlowEdge(\n", + " add_lines_on_diff_step, DIFF_WITH_LINES_IO, generate_comments_step, DIFF_WITH_LINES_IO\n", + " ),\n", + " DataFlowEdge(extract_file_path_step, FILEPATH_IO, generate_comments_step, FILEPATH_IO),\n", + " DataFlowEdge(\n", + " generate_comments_step,\n", + " JSON_COMMENTS_IO,\n", + " extract_comments_from_json_step,\n", + " JSON_COMMENTS_IO,\n", + " ),\n", + " ],\n", + ")\n", + "\n", + "# Use the MapStep to apply the sub flow to each file\n", + "for_each_file_step = MapStep(\n", + " flow=generate_comments_subflow,\n", + " unpack_input={\"message\": \".\"},\n", + " input_mapping={MapStep.ITERATED_INPUT: FILE_DIFF_LIST_IO},\n", + " output_descriptors=[\n", + " ListProperty(name=NESTED_COMMENT_LIST_IO, item_type=AnyProperty()),\n", + " ListProperty(name=FILEPATH_LIST_IO, item_type=StringProperty()),\n", + " ],\n", + " output_mapping={EXTRACTED_COMMENTS_IO: NESTED_COMMENT_LIST_IO, FILEPATH_IO: FILEPATH_LIST_IO},\n", + ")\n", + "\n", + "generate_all_comments_subflow = Flow.from_steps([for_each_file_step])" + ] + }, + { + "cell_type": "markdown", + "id": "d9da7ab6-0e38-416f-baf5-8a7e008a9abc", + "metadata": {}, + "source": [ + "**API Reference:** [Property](https://TODO/development/docs/api/flows.html#wayflowcore.property.Property) | [ListProperty](https://TODO/development/docs/api/flows.html#wayflowcore.property.ListProperty) | [DictProperty](https://TODO/development/docs/api/flows.html#wayflowcore.property.DictProperty) | [StringProperty](https://TODO/development/docs/api/flows.html#wayflowcore.property.StringProperty) |\n", + "[ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep) | [MapStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep) | [OutputMessageStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.outputmessagestep.OutputMessageStep) | [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep) | [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep)\n", + "\n", + "Take a look at each of the steps used in the sub-flow to get an understanding of what is happening.\n", + "\n", + "#### Format diff to string, `format_diff_to_string_step`\n", + "\n", + "This step converts the file diff list into a string so that it can be used by the following steps.\n", + "\n", + "This is done with the `string` Jinja filter as follows: `{{ message | string }}`. It uses an [OutputMessageStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.outputmessagestep.OutputMessageStep)\n", + "to achieve this.\n", + "\n", + "#### Add lines to the diff, `add_lines_on_diff_step`\n", + "\n", + "This step prefixes the diff with the line numbers required to review comments. It uses a, [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep), to run the\n", + "tool that you previously defined in order to do this.\n", + "\n", + "The input to the tool, within the I/O dictionary, is specified using the `input_mapping`. For all these steps, it is important to remember\n", + "that the outputs of one step are linked to the inputs of the next.\n", + "\n", + "#### Extract file path, `extract_file_path_step`\n", + "\n", + "This extracts the file path from the diff string. The file path is needed for assigning the review comments. The [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) step\n", + "is used to extract the file path from the diff.\n", + "\n", + "The regular expression is applied to the diff string, extracted form the input map using the ``input_mapping`` parameter.\n", + "\n", + "Note: Compared to the [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) used in Part 1, here only the first matchs is required.\n", + "\n", + "#### Generate comments, `generate_comments_step`\n", + "\n", + "This generates comments using the LLM and the prompt template defined earlier. The [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep) step executes\n", + "the prompt with the LLM defined earlier in this tutorial.\n", + "\n", + "Since the list of checks has already been defined, the template can be pre-rendered using the `render_template_partially` method. This renders the parts of the\n", + "template that have been provided, while the remaining information is gathered from the I/O dictionary.\n", + "\n", + "#### Extract comments from JSON, `extract_comments_from_json_step`\n", + "\n", + "This extracts the comments and line numbers from the generated LLM output, which is a serialized JSON structure due to the prompt used.\n", + "A [ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep) is used to do the extraction. When creating the step, specify the following in addition to the usual `input_mapping` and `output_mapping`:\n", + "\n", + "* `output_values`: This defines the [JQ](https://jqlang.github.io/jq/) query to extract the comments form the JSON generated by the LLM.\n", + "* `llms`: An LLM that can be used to help resolve any parsing errors. This is related to `retry`.\n", + "* `retry`: If parsing fails, you may want to retry. This is set to `True`, which results in trying to use the the LLM to help resolve any such issues.\n", + "\n", + "#### Create the sub-flow, `generate_comments_subflow`\n", + "\n", + "Here you define what steps are in the sub-flow, what the transitions between the steps are and what will be the starting step. This is exactly\n", + "the same process you did previously when defining the sub-flow to fetch the PR data.\n", + "\n", + "#### Applying the comment generation to all file diffs\n", + "\n", + "Now that you have the sub-flow create, you need to apply it to every file diff. This is done using a [MapStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep). `MapStep` takes a sub-flow as input, in this case, the `generate_comments_subflow`,\n", + "and applies it to an iterable—in this case, the list of file diffs.\n", + "\n", + "You simply specify:\n", + "\n", + "* `flow`: The sub-flow to map, that is applied to the iterable.\n", + "* `unpack_input`: Defines how to unpack the input. A [JQ](https://jqlang.github.io/jq/) query can be used to transform the input, but in this case, it is kept as a list.\n", + "* `input_mapping`: Defines what the sub-flow will iterate over. The key, [MapStep.ITERATED_INPUT](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep.ITERATED_INPUT), is used to pass in the diffs.\n", + "* `output_descriptors`: Specifies the values to collect from the output generated by applying the sub-flow. In this case, these will be the generated comments and the associated file path.\n", + "\n", + ".. note::\n", + " The :ref:`MapStep ` works similarly to how the Python map function works. For more information, see\n", + " https://docs.python.org/3/library/functions.html#map\n", + "\n", + "Finally, create the sub-flow to generate all comments using the helper method ``create_single_step_flow``.\n", + "\n", + "#### Testing the sub-flow\n", + "\n", + "You can test the sub-flow by creating a conversation, as shown in the code below, and specifying the inputs as done in, `Part 2: Retrieve the PR diff information`.\n", + "\n", + "Since each sub-flow is tested independently, you can reuse the output from the first sub-flow." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3af53c47-a721-4d03-8aed-e5a29ef2f4a0", + "metadata": {}, + "outputs": [], + "source": [ + "# we reuse the FILE_DIFF_LIST from the previous test\n", + "test_conversation = generate_all_comments_subflow.start_conversation(\n", + " inputs={\n", + " FILE_DIFF_LIST_IO: FILE_DIFF_LIST,\n", + " }\n", + ")\n", + "\n", + "execution_status = test_conversation.execute()\n", + "\n", + "assert isinstance(execution_status, FinishedStatus)\n", + "NESTED_COMMENT_LIST = execution_status.output_values[NESTED_COMMENT_LIST_IO]\n", + "FILEPATH_LIST = execution_status.output_values[FILEPATH_LIST_IO]\n", + "print(NESTED_COMMENT_LIST[0])\n", + "print(FILEPATH_LIST)" + ] + }, + { + "cell_type": "markdown", + "id": "29a32aea-66d3-4ee3-b6e2-1b8f20e219e5", + "metadata": {}, + "source": [ + "## Building the final Flow\n", + "\n", + "Congratulations! You have completed the three sub-flows, which, when combined into a single flow, will retrieve the PR diff information, generate comments on the diffs using an LLM.\n", + "\n", + "You will wire the sub-flows that you have built together by wrapping them in a [FlowExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.flowexecutionstep.FlowExecutionStep).\n", + "The [FlowExecutionSteps](https://TODO/development/docs/api/flows.html#wayflowcore.steps.flowexecutionstep.FlowExecutionStep) are then composed into the final combined Flow.\n", + "\n", + "The code for this is shown below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3045fddc-a43f-482e-a385-d5dc035ca0dd", + "metadata": {}, + "outputs": [], + "source": [ + "from wayflowcore.steps import FlowExecutionStep\n", + "\n", + "# Step Names\n", + "RETRIEVE_DIFF_FLOWSTEP = \"retrieve_diff_flowstep\"\n", + "GENERATE_COMMENTS_FLOWSTEP = \"generate_comments_flowstep\"\n", + "\n", + "# Steps\n", + "retrieve_diff_flowstep = FlowExecutionStep(flow=retrieve_diff_subflow)\n", + "generate_all_comments_flowstep = FlowExecutionStep(flow=generate_all_comments_subflow)\n", + "\n", + "pr_bot = Flow(\n", + " begin_step=retrieve_diff_flowstep,\n", + " steps={\n", + " RETRIEVE_DIFF_FLOWSTEP: retrieve_diff_flowstep,\n", + " GENERATE_COMMENTS_FLOWSTEP: generate_all_comments_flowstep,\n", + " },\n", + " control_flow_edges=[\n", + " ControlFlowEdge(retrieve_diff_flowstep, generate_all_comments_flowstep),\n", + " ControlFlowEdge(generate_all_comments_flowstep, None),\n", + " ],\n", + " data_flow_edges=[\n", + " DataFlowEdge(\n", + " retrieve_diff_flowstep,\n", + " FILE_DIFF_LIST_IO,\n", + " generate_all_comments_flowstep,\n", + " FILE_DIFF_LIST_IO,\n", + " )\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "9ff4f859-3a78-4b5b-a84a-a93ef7e1557a", + "metadata": {}, + "source": [ + "**API Reference:** [Flow](https://TODO/development/docs/api/flows.html#wayflowcore.flow.Flow) | [FlowExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.flowexecutionstep.FlowExecutionStep)" + ] + }, + { + "cell_type": "markdown", + "id": "de51a35e-29b9-402a-8632-90935e478331", + "metadata": {}, + "source": [ + "### Testing the combined assistant\n", + "\n", + "You can now run the PR bot end-to-end on your repo or locally.\n", + "\n", + "Set the `PATH_TO_DIR` to the actual path you extracted the sample codebase git repository to. You can also see how the output of the conversation\n", + "is extracted from the `execution_status` object, `execution_status.output_values`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39e5d773-e9af-4aea-acdc-174cdcd9eace", + "metadata": {}, + "outputs": [], + "source": [ + "conversation = pr_bot.start_conversation(inputs={REPO_DIRPATH_IO: PATH_TO_DIR})\n", + "\n", + "execution_status = conversation.execute()\n", + "\n", + "assert isinstance(execution_status, FinishedStatus)\n", + "print(execution_status.output_values)\n", + "\n", + "NESTED_COMMENT_LIST = execution_status.output_values[NESTED_COMMENT_LIST_IO]" + ] + }, + { + "cell_type": "markdown", + "id": "8e2ebfae-1c05-4965-8a98-b42a9f2d7371", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "In this tutorial you learned how to build a simple PR bot using WayFlow Flows, and learned:\n", + "\n", + "- How to use core steps such as the [OutputMessageStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.outputmessagestep.OutputMessageStep) and [PromptExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.promptexecutionstep.PromptExecutionStep).\n", + "- How to build and execute tools using the [ServerTool](https://TODO/development/docs/api/tools.html#wayflowcore.tools.servertools.ServerTool) and the [ToolExecutionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.toolexecutionstep.ToolExecutionStep).\n", + "- How to extract information using the [RegexExtractionStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep) and the [ExtractValueFromJsonStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep).\n", + "- How to apply a sub flow over an iterable data using the [MapStep](https://TODO/development/docs/api/flows.html#wayflowcore.steps.mapstep.MapStep).\n", + "\n", + "Finally, you learned how to structure code when building assistant as code and how to execute and combine sub flows to build complex assistant.\n", + "\n", + "This is an example of the kind of fully featured tool that you can build with WayFlow.\n", + "\n", + "## Next Steps\n", + "\n", + "Now that you learned how to build a PR reviewing assistant, you may want to check our other guides such as:\n", + "\n", + "- [Build a Simple Assistant with Agents](https://TODO/development/docs/tutorials/basic_agent.html).\n", + "- [How to Catch Exceptions in Flows](https://TODO/development/docs/howtoguides/catching_exceptions.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.21" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/26.1.2/_images/agentspec-icon.svg b/26.1.2/_images/agentspec-icon.svg new file mode 100644 index 000000000..3857dc814 --- /dev/null +++ b/26.1.2/_images/agentspec-icon.svg @@ -0,0 +1,46 @@ + + + + diff --git a/26.1.2/_images/agentstep.svg b/26.1.2/_images/agentstep.svg new file mode 100644 index 000000000..b6e8b530c --- /dev/null +++ b/26.1.2/_images/agentstep.svg @@ -0,0 +1,4 @@ + + + +
InputMessageStep
Writer agent
SendEmail
 diff --git a/26.1.2/_images/assistant_evaluator.png b/26.1.2/_images/assistant_evaluator.png new file mode 100644 index 000000000..24d2db1a3 Binary files /dev/null and b/26.1.2/_images/assistant_evaluator.png differ diff --git a/26.1.2/_images/branchingstep.svg b/26.1.2/_images/branchingstep.svg new file mode 100644 index 000000000..89103008a --- /dev/null +++ b/26.1.2/_images/branchingstep.svg @@ -0,0 +1,4 @@ + + + +
my_var == [FAILURE]
my_var == [FAILURE]
my_var == [SUCCESS]
my_var == [SUCCESS]
BranchingStep
RetryStep
Success
Success
Failure
Failure
my_var
my_var diff --git a/26.1.2/_images/catchexceptionstep.svg b/26.1.2/_images/catchexceptionstep.svg new file mode 100644 index 000000000..0d5eeae62 --- /dev/null +++ b/26.1.2/_images/catchexceptionstep.svg @@ -0,0 +1,4 @@ + + + +
...
...
...
...
Step B
Step B
Step A
Step A
Successful execution
Successful execution
Caught an Exception
Caught an Exception
CatchExceptionStep
CatchExceptionStep
Error-prone step
Error-prone step
sub-flow
sub-flow
Previous
steps
Previous...Text is not SVG - cannot display diff --git a/26.1.2/_images/conversation_evaluator.png b/26.1.2/_images/conversation_evaluator.png new file mode 100644 index 000000000..98bfbc2b7 Binary files /dev/null and b/26.1.2/_images/conversation_evaluator.png differ diff --git a/26.1.2/_images/dataflowedges_basics.jpg b/26.1.2/_images/dataflowedges_basics.jpg new file mode 100644 index 000000000..11f9dc9be Binary files /dev/null and b/26.1.2/_images/dataflowedges_basics.jpg differ diff --git a/26.1.2/_images/dataflowedges_looping.jpg b/26.1.2/_images/dataflowedges_looping.jpg new file mode 100644 index 000000000..624807508 Binary files /dev/null and b/26.1.2/_images/dataflowedges_looping.jpg differ diff --git a/26.1.2/_images/dataflowedges_routing.jpg b/26.1.2/_images/dataflowedges_routing.jpg new file mode 100644 index 000000000..8923fc316 Binary files /dev/null and b/26.1.2/_images/dataflowedges_routing.jpg differ diff --git a/26.1.2/_images/examples.svg b/26.1.2/_images/examples.svg new file mode 100644 index 000000000..26c974207 --- /dev/null +++ b/26.1.2/_images/examples.svg @@ -0,0 +1,4 @@ + + + + diff --git a/26.1.2/_images/exploring.svg b/26.1.2/_images/exploring.svg new file mode 100644 index 000000000..9b535336f --- /dev/null +++ b/26.1.2/_images/exploring.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/26.1.2/_images/extensibility.svg b/26.1.2/_images/extensibility.svg new file mode 100644 index 000000000..3652ec783 --- /dev/null +++ b/26.1.2/_images/extensibility.svg @@ -0,0 +1,3 @@ + + + diff --git a/26.1.2/_images/flexibility.svg b/26.1.2/_images/flexibility.svg new file mode 100644 index 000000000..0b7b05063 --- /dev/null +++ b/26.1.2/_images/flexibility.svg @@ -0,0 +1,3 @@ + + + diff --git a/26.1.2/_images/hierarchical_vs_swarm.svg b/26.1.2/_images/hierarchical_vs_swarm.svg new file mode 100644 index 000000000..e5240987c --- /dev/null +++ b/26.1.2/_images/hierarchical_vs_swarm.svg @@ -0,0 +1,4 @@ + + + +
Hierarchical pattern
Hierarchical pattern
~ 5 LLM calls
for the route
User → Agent K → User
~ 5 LLM calls...
Agent K
Agent K
Human
User
Human...
~ 3 LLM calls 
(2 with handoff) for the route
User → Agent K → User
~ 3 LLM calls...
Agent K
Agent K
Human
User
Human...
Swarm pattern
Swarm patternText is not SVG - cannot display diff --git a/26.1.2/_images/howto_multiagent.svg b/26.1.2/_images/howto_multiagent.svg new file mode 100644 index 000000000..44f830b05 --- /dev/null +++ b/26.1.2/_images/howto_multiagent.svg @@ -0,0 +1,4 @@ + + + +
Customer service manager
Customer service m...
Refund specialist
Refund specialist
Satisfaction surveyor
Satisfaction surveyor
answers
answers
queries
queries
User
UserText is not SVG - cannot display diff --git a/26.1.2/_images/interoperability.svg b/26.1.2/_images/interoperability.svg new file mode 100644 index 000000000..3bb34e45a --- /dev/null +++ b/26.1.2/_images/interoperability.svg @@ -0,0 +1,3 @@ + + + diff --git a/26.1.2/_images/mapstep.svg b/26.1.2/_images/mapstep.svg new file mode 100644 index 000000000..19d6d34d2 --- /dev/null +++ b/26.1.2/_images/mapstep.svg @@ -0,0 +1,4 @@ + + + +
...
...
M
A
P
MapStep
sub-flow
Previous
steps
R
E
D
U
C
E
Next
steps
 diff --git a/26.1.2/_images/ollama_extract1_2.png b/26.1.2/_images/ollama_extract1_2.png new file mode 100644 index 000000000..582f4c93e Binary files /dev/null and b/26.1.2/_images/ollama_extract1_2.png differ diff --git a/26.1.2/_images/ollama_extract2_2.png b/26.1.2/_images/ollama_extract2_2.png new file mode 100644 index 000000000..a3dc97b49 Binary files /dev/null and b/26.1.2/_images/ollama_extract2_2.png differ diff --git a/26.1.2/_images/ollama_frontpage.png b/26.1.2/_images/ollama_frontpage.png new file mode 100644 index 000000000..73ad24cfd Binary files /dev/null and b/26.1.2/_images/ollama_frontpage.png differ diff --git a/26.1.2/_images/ollama_in_terminal.png b/26.1.2/_images/ollama_in_terminal.png new file mode 100644 index 000000000..243f0d85d Binary files /dev/null and b/26.1.2/_images/ollama_in_terminal.png differ diff --git a/26.1.2/_images/ollama_install1_3.png b/26.1.2/_images/ollama_install1_3.png new file mode 100644 index 000000000..767f442ae Binary files /dev/null and b/26.1.2/_images/ollama_install1_3.png differ diff --git a/26.1.2/_images/ollama_install2_3.png b/26.1.2/_images/ollama_install2_3.png new file mode 100644 index 000000000..dea0521d5 Binary files /dev/null and b/26.1.2/_images/ollama_install2_3.png differ diff --git a/26.1.2/_images/ollama_install3_3.png b/26.1.2/_images/ollama_install3_3.png new file mode 100644 index 000000000..cd1f0a0c7 Binary files /dev/null and b/26.1.2/_images/ollama_install3_3.png differ diff --git a/26.1.2/_images/opennes.svg b/26.1.2/_images/opennes.svg new file mode 100644 index 000000000..b11d8241d --- /dev/null +++ b/26.1.2/_images/opennes.svg @@ -0,0 +1,3 @@ + + + diff --git a/26.1.2/_images/prbot_generate_comment.svg b/26.1.2/_images/prbot_generate_comment.svg new file mode 100644 index 000000000..66ccc9a19 --- /dev/null +++ b/26.1.2/_images/prbot_generate_comment.svg @@ -0,0 +1,4 @@ + + + +

Format diff
to string
Add lines
on diff
Extract diff
file path
Generate
comments
Extract comments
from json
OutputMessageStep
ToolExecutionStep
RegexExtractionStep
PromptExecutionStep
ExtractValueFromJsonStep
Get diff info
Post 
comments
MapStep
 diff --git a/26.1.2/_images/prbot_main.svg b/26.1.2/_images/prbot_main.svg new file mode 100644 index 000000000..828d08766 --- /dev/null +++ b/26.1.2/_images/prbot_main.svg @@ -0,0 +1,4 @@ + + + +

Extract list
of file diffs


Generate
comments
Get Code
diffs

Step 1.

Raw code diff info is retrieved from a local git repository

Step 2.

A list of unit diff info is created from the raw code diffs

Step 3.

For each unit diff, a list of comments is created
Get diff info
Review diffs
Local Code Repo
Generated Comments
 diff --git a/26.1.2/_images/prbot_retrieve_diffs.svg b/26.1.2/_images/prbot_retrieve_diffs.svg new file mode 100644 index 000000000..e578e2063 --- /dev/null +++ b/26.1.2/_images/prbot_retrieve_diffs.svg @@ -0,0 +1,4 @@ + + + +

Get raw
code diffs
Extract into
list of file diffs
ToolExecutionStep
RegexExtractionStep
Review diffs
START
Local Code Repo
 diff --git a/26.1.2/_images/python-icon.svg b/26.1.2/_images/python-icon.svg new file mode 100644 index 000000000..11c0e4dc5 --- /dev/null +++ b/26.1.2/_images/python-icon.svg @@ -0,0 +1,2 @@ + + diff --git a/26.1.2/_images/reusability.svg b/26.1.2/_images/reusability.svg new file mode 100644 index 000000000..8aebf6fed --- /dev/null +++ b/26.1.2/_images/reusability.svg @@ -0,0 +1,3 @@ + + + diff --git a/26.1.2/_images/ser_deser.svg b/26.1.2/_images/ser_deser.svg new file mode 100644 index 000000000..65940e5d5 --- /dev/null +++ b/26.1.2/_images/ser_deser.svg @@ -0,0 +1,4 @@ + + + +
_component_type: Flow
_referenced_objects:
  ...
steps:
  ...
transitions:
  ...
_component_type: Flow...
serialize()
serialize()
Agent
Agent
Flow
Flow
creates
creates
_component_type: Flow
_referenced_objects:
  ...
steps:
  ...
transitions:
  ...
_component_type: Flow...
autodeserialize()
autodeserialize()
Agent
Agent
Flow
Flow
execute
agent / flow
execute...
serialization
serialization
deserialization
deserializat...Text is not SVG - cannot display diff --git a/26.1.2/_images/swarm_example.svg b/26.1.2/_images/swarm_example.svg new file mode 100644 index 000000000..4bf220314 --- /dev/null +++ b/26.1.2/_images/swarm_example.svg @@ -0,0 +1,4 @@ + + + +
User
General Practitioner Doctor
Pharmacist
Dermatologist
Symptoms Knowledge
Skin conditions Knowledge
Medications Knowledge
symptom checker tool
medication info tool
skin condition info tool
queries
answers
queries
answers
queries
answers
queries
answers
 diff --git a/26.1.2/_images/tutorials.svg b/26.1.2/_images/tutorials.svg new file mode 100644 index 000000000..2f825bba7 --- /dev/null +++ b/26.1.2/_images/tutorials.svg @@ -0,0 +1,10 @@ + + + + + + + + + + diff --git a/26.1.2/_images/types_of_tools.svg b/26.1.2/_images/types_of_tools.svg new file mode 100644 index 000000000..d6a96130f --- /dev/null +++ b/26.1.2/_images/types_of_tools.svg @@ -0,0 +1,4 @@ + + + +
Agent / LLM application

eg, OCI AI Agent Platform
Client application

ex: Local machine or browser
API provider

ex: OCI Function, MS Cognitive Services, AWS Lamba
End-user
Tools executing here and invoked by WayFlow are
"remote" 
Tools executing here are
"server-side" 
Tools executing here are
"client-side" 
This includes remote APIs invoked from the Client application
 diff --git a/26.1.2/_sources/core/_components/llm_config_tabs.rst.txt b/26.1.2/_sources/core/_components/llm_config_tabs.rst.txt new file mode 100644 index 000000000..0d906d008 --- /dev/null +++ b/26.1.2/_sources/core/_components/llm_config_tabs.rst.txt @@ -0,0 +1,40 @@ +.. :orphan: + +.. :no-search: + +.. tabs:: + + .. tab:: OCI GenAI + + .. code-block:: python + + from wayflowcore.models import OCIGenAIModel, OCIClientConfigWithApiKey + + llm = OCIGenAIModel( + model_id="provider.model-id", + compartment_id="compartment-id", + client_config=OCIClientConfigWithApiKey( + service_endpoint="https://url-to-service-endpoint.com", + ), + ) + + .. tab:: vLLM + + .. code-block:: python + + from wayflowcore.models import VllmModel + + llm = VllmModel( + model_id="model-id", + host_port="VLLM_HOST_PORT", + ) + + .. tab:: Ollama + + .. code-block:: python + + from wayflowcore.models import OllamaModel + + llm = OllamaModel( + model_id="model-id", + ) diff --git a/26.1.2/_sources/core/api/a2a.rst.txt b/26.1.2/_sources/core/api/a2a.rst.txt new file mode 100644 index 000000000..9ae5e41c4 --- /dev/null +++ b/26.1.2/_sources/core/api/a2a.rst.txt @@ -0,0 +1,11 @@ +A2A related classes +=================== + +.. autoclass:: wayflowcore.a2a.a2aagent.A2AConnectionConfig + +.. autoclass:: wayflowcore.a2a.a2aagent.A2ASessionParameters + +A2A Agent +--------- + +refer :ref:`A2AAgent ` diff --git a/26.1.2/_sources/core/api/agent.rst.txt b/26.1.2/_sources/core/api/agent.rst.txt new file mode 100644 index 000000000..fc5fc6663 --- /dev/null +++ b/26.1.2/_sources/core/api/agent.rst.txt @@ -0,0 +1,80 @@ +Agents +====== + +This page presents all APIs and classes related to WayFlow agents. + + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/agent.html + :link-alt: Agent Spec - Agents API Reference + + Visit the Agent Spec API Documentation to learn more about Agent Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + +Agent related classes +--------------------- + +Agent class +~~~~~~~~~~~ + +.. _agent: +.. autoclass:: wayflowcore.agent.Agent + :exclude-members: execute + +OCI Agent class +~~~~~~~~~~~~~~~ + +.. _ociagent: +.. autoclass:: wayflowcore.ociagent.OciAgent + :exclude-members: execute + +A2A Agent class +~~~~~~~~~~~~~~~ + +.. _a2aagent: +.. autoclass:: wayflowcore.a2a.a2aagent.A2AAgent + :exclude-members: execute + +DescribedFlow +~~~~~~~~~~~~~ + +.. _describedflow: +.. autoclass:: wayflowcore.tools.DescribedFlow + +DescribedAgent +~~~~~~~~~~~~~~ + +.. _describedassistant: +.. autoclass:: wayflowcore.tools.DescribedAgent + +Swarm class +~~~~~~~~~~~ + +.. _swarm: +.. autoclass:: wayflowcore.swarm.Swarm + +.. _handoffmode: +.. autoclass:: wayflowcore.swarm.HandoffMode + +ManagerWorkers class +~~~~~~~~~~~~~~~~~~~~ + +.. _managerworkers: +.. autoclass:: wayflowcore.managerworkers.ManagerWorkers + + +Agent Behavior Configuration +---------------------------- + +.. _callerinputmode: +.. autoclass:: wayflowcore.agent.CallerInputMode diff --git a/26.1.2/_sources/core/api/agentserver.rst.txt b/26.1.2/_sources/core/api/agentserver.rst.txt new file mode 100644 index 000000000..92ba75b4c --- /dev/null +++ b/26.1.2/_sources/core/api/agentserver.rst.txt @@ -0,0 +1,32 @@ +Agent Server +============ + +This page covers the components used to expose WayFlow agents through different protocols. + + +Server application +------------------ + +.. _openairesponsesserver: +.. autoclass:: wayflowcore.agentserver.server.OpenAIResponsesServer + :members: __init__, get_app, run + +.. _a2aserver: +.. autoclass:: wayflowcore.agentserver.server.A2AServer + :members: __init__, get_app, run + +.. _serverstorageconfig: +.. autoclass:: wayflowcore.agentserver.serverstorageconfig.ServerStorageConfig + :members: to_schema + +.. autofunction:: wayflowcore.agentserver.app.create_server_app + + +CLI Reference +------------- + +.. _cliwayflowreference: +.. argparse:: + :module: wayflowcore.cli + :func: build_parser + :prog: wayflow diff --git a/26.1.2/_sources/core/api/agentspec.rst.txt b/26.1.2/_sources/core/api/agentspec.rst.txt new file mode 100644 index 000000000..6bc75ae78 --- /dev/null +++ b/26.1.2/_sources/core/api/agentspec.rst.txt @@ -0,0 +1,292 @@ +.. _agentspec_adapters: + +Agent Spec Adapters +=================== + +This page presents all APIs and classes related to Agent Spec and WayFlow. + + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/index.html + :link-alt: Agent Spec - API Reference + + Visit the Agent Spec API Documentation to learn more about the native Agent Spec Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + + +.. _agentspecexporter: +.. autoclass:: wayflowcore.agentspec.agentspecexporter.AgentSpecExporter + + +.. _agentspecloader: +.. autoclass:: wayflowcore.agentspec.runtimeloader.AgentSpecLoader + + +Agent Spec Tracing +================== + +This event listener makes WayFlow components emit traces according to the Agent Spec Tracing standard. + +.. _agentspeceventlistener: +.. autoclass:: wayflowcore.agentspec.tracing.AgentSpecEventListener + + +Custom Components +================= + +These are example of custom Agent Spec components that can be used in Agent Spec configurations and +loaded/executed in WayFlow. + +.. note:: + Both extended and plugin components are introduced to allow assistant developers to export + their WayFlow assistants to Agent Spec. + + They may be added as native Agent Spec components with modified component name and fields. + +Extended Components +------------------- + +Extended components are Agent Spec components extended with additional fields. + +.. _agentspecagent: +.. autoclass:: wayflowcore.agentspec.components.agent.ExtendedAgent + :exclude-members: model_post_init, model_config + +.. _agentspecflow: +.. autoclass:: wayflowcore.agentspec.components.flow.ExtendedFlow + :exclude-members: model_post_init, model_config + +.. _agentspectoolnode: +.. autoclass:: wayflowcore.agentspec.components.nodes.ExtendedToolNode + :exclude-members: model_post_init, model_config + +.. _agentspecllmnode: +.. autoclass:: wayflowcore.agentspec.components.nodes.ExtendedLlmNode + :exclude-members: model_post_init, model_config + +.. _agentspecmapnode: +.. autoclass:: wayflowcore.agentspec.components.nodes.ExtendedMapNode + :exclude-members: model_post_init, model_config + + +Plugin Components +----------------- + +Plugin components are new components that are not natively supported in Agent Spec. + + +Agentic patterns +^^^^^^^^^^^^^^^^ + +.. _agentspecswarmpattern: +.. autoclass:: wayflowcore.agentspec.components.swarm.PluginSwarm + :exclude-members: model_post_init, model_config + +Tools +^^^^^ + +.. _agentspectoolfromtoolbox: +.. autoclass:: wayflowcore.agentspec.components.tools.PluginToolFromToolBox + :exclude-members: model_post_init, model_config + +Search +^^^^^^ + +.. _agentspecsearchtoolbox: +.. autoclass:: wayflowcore.agentspec.components.search.PluginSearchToolBox + :exclude-members: model_post_init, model_config + +.. _agentspecvectorretrieverconfig: +.. autoclass:: wayflowcore.agentspec.components.search.PluginVectorRetrieverConfig + :exclude-members: model_post_init, model_config + +.. _agentspecsearchconfig: +.. autoclass:: wayflowcore.agentspec.components.search.PluginSearchConfig + :exclude-members: model_post_init, model_config + +.. _agentspecvectorconfig: +.. autoclass:: wayflowcore.agentspec.components.search.PluginVectorConfig + :exclude-members: model_post_init, model_config + +.. _agentspecoracledatabasedatastore: +.. autoclass:: wayflowcore.agentspec.components.datastores.oracle_datastore.PluginOracleDatabaseDatastore + :exclude-members: model_post_init, model_config + + +Embedding Models +^^^^^^^^^^^^^^^^ + +.. _agentspec_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginEmbeddingConfig + :exclude-members: model_post_init, model_config + +.. _agentspec_oci_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginOciGenAiEmbeddingConfig + :exclude-members: model_post_init, model_config + +.. _agentspec_openai_compatible_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginOpenAiCompatibleEmbeddingConfig + :exclude-members: model_post_init, model_config + +.. _agentspec_ollama_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginOllamaEmbeddingConfig + :exclude-members: model_post_init, model_config + +.. _agentspec_openai_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginOpenAiEmbeddingConfig + :exclude-members: model_post_init, model_config + +.. _agentspec_vllm_embedding_model_config: +.. autoclass:: wayflowcore.agentspec.components.embeddingmodels.PluginVllmEmbeddingConfig + :exclude-members: model_post_init, model_config + +Messages +^^^^^^^^ + +.. _agentspecmessage: +.. autoclass:: wayflowcore.agentspec.components.messagelist.PluginMessage + :exclude-members: model_post_init, model_config + +.. _agentspectextcontent: +.. autoclass:: wayflowcore.agentspec.components.messagelist.PluginTextContent + :exclude-members: model_post_init, model_config + +.. _agentspecimagecontent: +.. autoclass:: wayflowcore.agentspec.components.messagelist.PluginImageContent + :exclude-members: model_post_init, model_config + +.. _agentspecregexpattern: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginRegexPattern + :exclude-members: model_post_init, model_config + +.. _agentspecoutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginOutputParser + +.. _agentspecregexoutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginRegexOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspecjsonoutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginJsonOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspectooloutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginToolOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspecjsontooloutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginJsonToolOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspecpythontooloutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginPythonToolOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspecreacttooloutputparser: +.. autoclass:: wayflowcore.agentspec.components.outputparser.PluginReactToolOutputParser + :exclude-members: model_post_init, model_config + +.. _agentspecprompttemplate: +.. autoclass:: wayflowcore.agentspec.components.template.PluginPromptTemplate + :exclude-members: model_post_init, model_config + +.. _agentspeccoalescesystemmessagestransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginCoalesceSystemMessagesTransform + :exclude-members: model_post_init, model_config + +.. _agentspecremoveemptynonusermessagetransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginRemoveEmptyNonUserMessageTransform + :exclude-members: model_post_init, model_config + +.. _agentspecappendtrailingsystemmessagetousermessagetransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginAppendTrailingSystemMessageToUserMessageTransform + :exclude-members: model_post_init, model_config + +.. _agentspecllamamergetoolrequestsandcallstransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginLlamaMergeToolRequestAndCallsTransform + :exclude-members: model_post_init, model_config + +.. _agentspecreactmergetoolrequestsandcallstransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginReactMergeToolRequestAndCallsTransform + :exclude-members: model_post_init, model_config + +.. _agentspeccanonicalizationmessagetransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginCanonicalizationMessageTransform + :exclude-members: model_post_init, model_config + +.. _agentspecsplitpromptonmarkermessagetransform: +.. autoclass:: wayflowcore.agentspec.components.transforms.PluginSplitPromptOnMarkerMessageTransform + :exclude-members: model_post_init, model_config + +Nodes +^^^^^ + +.. _agentspeccatchexceptionnode: +.. autoclass:: wayflowcore.agentspec.components.nodes.PluginCatchExceptionNode + :exclude-members: model_post_init, model_config + +.. _agentspecextractnode: +.. autoclass:: wayflowcore.agentspec.components.nodes.PluginExtractNode + :exclude-members: model_post_init, model_config + +.. _agentspecinputmessagenode: +.. autoclass:: wayflowcore.agentspec.components.nodes.PluginInputMessageNode + :exclude-members: model_post_init, model_config + +.. _agentspecoutputmessagenode: +.. autoclass:: wayflowcore.agentspec.components.nodes.PluginOutputMessageNode + :exclude-members: model_post_init, model_config + +.. _agentspecdatastorecreatenode: +.. autoclass:: wayflowcore.agentspec.components.datastores.nodes.PluginDatastoreCreateNode + :exclude-members: model_post_init, model_config + +.. _agentspecdatastoredeletenode: +.. autoclass:: wayflowcore.agentspec.components.datastores.nodes.PluginDatastoreDeleteNode + :exclude-members: model_post_init, model_config + +.. _agentspecdatastorelistnode: +.. autoclass:: wayflowcore.agentspec.components.datastores.nodes.PluginDatastoreListNode + :exclude-members: model_post_init, model_config + +.. _agentspecdatastorequerynode: +.. autoclass:: wayflowcore.agentspec.components.datastores.nodes.PluginDatastoreQueryNode + :exclude-members: model_post_init, model_config + +.. _agentspecdatastoreupdatenode: +.. autoclass:: wayflowcore.agentspec.components.datastores.nodes.PluginDatastoreUpdateNode + :exclude-members: model_post_init, model_config + +Context Providers +^^^^^^^^^^^^^^^^^ + +.. _agentspeccontextprovider: +.. autoclass:: wayflowcore.agentspec.components.contextprovider.PluginContextProvider + :exclude-members: model_post_init, model_config + +Tools +^^^^^ + +.. _agentspecplugintoolrequest: +.. autoclass:: wayflowcore.agentspec.components.tools.PluginToolRequest + +.. _agentspecplugintoolresult: +.. autoclass:: wayflowcore.agentspec.components.tools.PluginToolResult + + +Context +^^^^^^^ + +.. autoclass:: wayflowcore.agentspec._runtimeconverter.AgentSpecToWayflowConversionContext + +.. autoclass:: wayflowcore.agentspec._agentspecconverter.WayflowToAgentSpecConversionContext diff --git a/26.1.2/_sources/core/api/auth.rst.txt b/26.1.2/_sources/core/api/auth.rst.txt new file mode 100644 index 000000000..e48693e5f --- /dev/null +++ b/26.1.2/_sources/core/api/auth.rst.txt @@ -0,0 +1,36 @@ +Auth +==== + +This page presents all APIs and classes related to WayFlow Auth configuration components. + + +Auth classes +------------ + +.. _authconfig: +.. autoclass:: wayflowcore.auth.auth.AuthConfig + +.. _oauthendpoints: +.. autoclass:: wayflowcore.auth.auth.OAuthEndpoints + +.. _pkcemethod: +.. autoclass:: wayflowcore.auth.auth.PKCEMethod + +.. _pkcepolicy: +.. autoclass:: wayflowcore.auth.auth.PKCEPolicy + +.. _oauthclientconfig: +.. autoclass:: wayflowcore.auth.auth.OAuthClientConfig + +.. _oauthconfig: +.. autoclass:: wayflowcore.auth.auth.OAuthConfig + + +Auth-related Classes +-------------------- + +.. _authchallengerequest: +.. autoclass:: wayflowcore.auth.auth.AuthChallengeRequest + +.. _authchallengeresult: +.. autoclass:: wayflowcore.auth.auth.AuthChallengeResult diff --git a/26.1.2/_sources/core/api/component.rst.txt b/26.1.2/_sources/core/api/component.rst.txt new file mode 100644 index 000000000..2bf481413 --- /dev/null +++ b/26.1.2/_sources/core/api/component.rst.txt @@ -0,0 +1,5 @@ +Component +========= + +.. _component: +.. autoclass:: wayflowcore.component.Component diff --git a/26.1.2/_sources/core/api/contextproviders.rst.txt b/26.1.2/_sources/core/api/contextproviders.rst.txt new file mode 100644 index 000000000..f1381bc02 --- /dev/null +++ b/26.1.2/_sources/core/api/contextproviders.rst.txt @@ -0,0 +1,17 @@ +Context Providers +================= + +Base classes +------------ + +.. _contextprovider: +.. autoclass:: wayflowcore.contextproviders.contextprovider.ContextProvider + +Available Context Providers +--------------------------- + +.. _toolcontextprovider: +.. autoclass:: wayflowcore.contextproviders.toolcontextprovider.ToolContextProvider + +.. _flowcontextprovider: +.. autoclass:: wayflowcore.contextproviders.flowcontextprovider.FlowContextProvider diff --git a/26.1.2/_sources/core/api/conversation.rst.txt b/26.1.2/_sources/core/api/conversation.rst.txt new file mode 100644 index 000000000..fc70aba07 --- /dev/null +++ b/26.1.2/_sources/core/api/conversation.rst.txt @@ -0,0 +1,67 @@ +Conversations +============= + +Messages +-------- + +.. _message: +.. autoclass:: wayflowcore.messagelist.Message + +.. _messagetype: +.. autoclass:: wayflowcore.messagelist.MessageType + +.. _messagecontent: +.. autoclass:: wayflowcore.messagelist.MessageContent + +.. _textcontent: +.. autoclass:: wayflowcore.messagelist.TextContent + +.. _texttokenlogprob: +.. autoclass:: wayflowcore.messagelist.TextTokenLogProb + +.. _texttokentoplogprob: +.. autoclass:: wayflowcore.messagelist.TextTokenTopLogProb + +.. _imagecontent: +.. autoclass:: wayflowcore.messagelist.ImageContent + +.. _messagelist: +.. autoclass:: wayflowcore.messagelist.MessageList + +ExecutionStatus +--------------- + +.. _executionstatus: +.. autoclass:: wayflowcore.executors.executionstatus.ExecutionStatus + +.. _usermessagerequesttatus: +.. autoclass:: wayflowcore.executors.executionstatus.UserMessageRequestStatus + +.. _finishedexecutionstatus: +.. autoclass:: wayflowcore.executors.executionstatus.FinishedStatus + +.. _toolrequestexecutionstatus: +.. autoclass:: wayflowcore.executors.executionstatus.ToolRequestStatus + +.. _authchallengerequeststatus: +.. autoclass:: wayflowcore.executors.executionstatus.AuthChallengeRequestStatus + +Conversation +------------ + +Base class for conversations. Can manipulate a conversation object, and can be serialized/deserialized. + +.. _conversationalcomponent: +.. autoclass:: wayflowcore.conversationalcomponent.ConversationalComponent + +.. _conversation: +.. autoclass:: wayflowcore.conversation.Conversation + +Execution Plan +-------------- + +.. autoclass:: wayflowcore.planning.ExecutionPlan + +.. autoclass:: wayflowcore.planning.Task + +.. autoclass:: wayflowcore.planning.TaskStatus diff --git a/26.1.2/_sources/core/api/datastores.rst.txt b/26.1.2/_sources/core/api/datastores.rst.txt new file mode 100644 index 000000000..327e8da67 --- /dev/null +++ b/26.1.2/_sources/core/api/datastores.rst.txt @@ -0,0 +1,86 @@ +.. _datastores: + +Datastores +========== + +Datastores connect Agents and Flows to data modelled by entity definitions. +To see how you can use a datastore in a Flow or Agent, see :ref:`Datastore task steps `. + +Entity +------ + +Entities define the data model of datastores. + +.. _entity: +.. autoclass:: wayflowcore.datastore.Entity + +.. _nullable_helper_function: +.. autoclass:: wayflowcore.datastore.nullable + +Base class +---------- + +.. _datastore: +.. autoclass:: wayflowcore.datastore.Datastore + +In memory +--------- + +.. _inmemorydatastore: +.. autoclass:: wayflowcore.datastore.InMemoryDatastore + :exclude-members: list, create, update, delete, describe + + .. + We exclude these members because they are already included in the base class + +Relational Datastore +-------------------- + +.. autoclass:: wayflowcore.datastore._relational.RelationalDatastore + +Oracle Database +--------------- + +.. important:: + The Oracle Database Datastore requires additional optional dependencies, which can be installed + with the ``[datastore]`` installation option. + +.. note:: + By default (when using the `OracleDatabaseConnectionConfig` classes as-is), the `python-oracledb` + client will use a thin connection to the database. If you want to use a thick connection + (leveraging Oracle Instant Client), invoke `oracledb.init_instant_client()` before initializing + any connection to the database. More information about thick and thin connection can be found in the + `python-oracledb documentation `_. + +.. _oracledatabaseconnectionconfig: +.. autoclass:: wayflowcore.datastore.OracleDatabaseConnectionConfig + +.. _oracledatabasetlsconnectionconfig: +.. autoclass:: wayflowcore.datastore.TlsOracleDatabaseConnectionConfig + +.. _oracledatabasemtlsconnectionconfig: +.. autoclass:: wayflowcore.datastore.MTlsOracleDatabaseConnectionConfig + +.. _oracledatabasedatastore: +.. autoclass:: wayflowcore.datastore.OracleDatabaseDatastore + + .. automethod:: query + + +Postgres Database +----------------- + +.. important:: + The Postgres Database Datastore requires additional optional dependencies, which can be installed + with the ``[datastore]`` installation option. + +.. _postgresdatabaseconnectionconfig: +.. autoclass:: wayflowcore.datastore.PostgresDatabaseConnectionConfig + +.. _postgresdatabasetlsconnectionconfig: +.. autoclass:: wayflowcore.datastore.TlsPostgresDatabaseConnectionConfig + +.. _postgresdatabasedatastore: +.. autoclass:: wayflowcore.datastore.PostgresDatabaseDatastore + + .. automethod:: query diff --git a/26.1.2/_sources/core/api/embeddingmodels.rst.txt b/26.1.2/_sources/core/api/embeddingmodels.rst.txt new file mode 100644 index 000000000..a63cf0d6c --- /dev/null +++ b/26.1.2/_sources/core/api/embeddingmodels.rst.txt @@ -0,0 +1,43 @@ +Embedding Models +================ + +This page presents all APIs and classes related to LLM Embedding models. + +EmbeddingModel +----------------- + +.. _embeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.embeddingmodel.EmbeddingModel + +All models +---------- + +OpenAI Compatible Embedding Models +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _openaicompatibleembeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.openaicompatiblemodel.OpenAICompatibleEmbeddingModel + +OpenAI Embedding Models +~~~~~~~~~~~~~~~~~~~~~~~ + +.. _openaiembeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.openaimodel.OpenAIEmbeddingModel + +VLLM Embedding Models +~~~~~~~~~~~~~~~~~~~~~ + +.. _vllmembeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.vllmmodel.VllmEmbeddingModel + +OCI GenAI Embedding Models +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _ocigenaiembeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.ocigenaimodel.OCIGenAIEmbeddingModel + +Ollama Embedding Models +~~~~~~~~~~~~~~~~~~~~~~~ + +.. _ollamaembeddingmodel: +.. autoclass:: wayflowcore.embeddingmodels.ollamamodel.OllamaEmbeddingModel \ No newline at end of file diff --git a/26.1.2/_sources/core/api/evaluation.rst.txt b/26.1.2/_sources/core/api/evaluation.rst.txt new file mode 100644 index 000000000..412e5f688 --- /dev/null +++ b/26.1.2/_sources/core/api/evaluation.rst.txt @@ -0,0 +1,59 @@ +.. _evaluation: + +Evaluation APIs +=============== + + +Assistant Evaluation +-------------------- + +.. _evaluationtask: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.EvaluationTask + +.. _evaluationenvironment: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.EvaluationEnvironment + +.. _assistantevaluator: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.AssistantEvaluator + +.. _taskscorer: +.. autoclass:: wayflowcore.evaluation.taskscorer.TaskScorer + +.. _runproxyconversation: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.run_proxy_agent_conversation + +.. _assistantevaluationresult: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.AssistantEvaluationResult + +.. _humanproxyassistant: +.. autoclass:: wayflowcore.evaluation.assistantevaluator.HumanProxyAssistant + +Conversation Evaluation +----------------------- + +.. _conversationevaluator: +.. autoclass:: wayflowcore.evaluation.conversationevaluator.ConversationEvaluator + +.. _conversationscorer: +.. autoclass:: wayflowcore.evaluation.conversationscorer.ConversationScorer + +.. _usefullnessscorer: +.. autoclass:: wayflowcore.evaluation.usefulnessscorer.UsefulnessScorer + +.. _userhappinessscorer: +.. autoclass:: wayflowcore.evaluation.userhappinessscorer.UserHappinessScorer + +.. _criteriascorer: +.. autoclass:: wayflowcore.evaluation.criteriascorer.CriteriaScorer + +.. _defaultscoremap: +.. autoclass:: wayflowcore.evaluation.criteriascorer.DEFAULT_SCORE_MAP + +Evaluation Metrics +------------------ + +.. _calculate_set_metrics: +.. autofunction:: wayflowcore.evaluation.evaluation_metrics.calculate_set_metrics + +.. _calculate_accuracy: +.. autofunction:: wayflowcore.evaluation.evaluation_metrics.calculate_accuracy diff --git a/26.1.2/_sources/core/api/events.rst.txt b/26.1.2/_sources/core/api/events.rst.txt new file mode 100644 index 000000000..cd3996709 --- /dev/null +++ b/26.1.2/_sources/core/api/events.rst.txt @@ -0,0 +1,143 @@ +.. _events: + +Events +====== + +This page presents all APIs and classes related to events in WayFlow. + +Events +------ + +.. _event: +.. autoclass:: wayflowcore.events.event.Event + +.. _llmgenerationrequestevent: +.. autoclass:: wayflowcore.events.event.LlmGenerationRequestEvent + :exclude-members: to_tracing_info + +.. _llmgenerationresponseevent: +.. autoclass:: wayflowcore.events.event.LlmGenerationResponseEvent + :exclude-members: to_tracing_info + +.. _conversationalcomponentexecutionstartedevent: +.. autoclass:: wayflowcore.events.event.ConversationalComponentExecutionStartedEvent + :exclude-members: to_tracing_info + +.. _conversationalcomponentexecutionfinishedevent: +.. autoclass:: wayflowcore.events.event.ConversationalComponentExecutionFinishedEvent + :exclude-members: to_tracing_info + +.. _conversationcreatedevent: +.. autoclass:: wayflowcore.events.event.ConversationCreatedEvent + :exclude-members: to_tracing_info + +.. _conversationmessageaddedevent: +.. autoclass:: wayflowcore.events.event.ConversationMessageAddedEvent + :exclude-members: to_tracing_info + +.. _conversationmessagestreamstartedevent: +.. autoclass:: wayflowcore.events.event.ConversationMessageStreamStartedEvent + :exclude-members: to_tracing_info + +.. _conversationmessagestreamchunkevent: +.. autoclass:: wayflowcore.events.event.ConversationMessageStreamChunkEvent + :exclude-members: to_tracing_info + +.. _conversationmessagestreamendedevent: +.. autoclass:: wayflowcore.events.event.ConversationMessageStreamEndedEvent + :exclude-members: to_tracing_info + +.. _conversationexecutionstartedevent: +.. autoclass:: wayflowcore.events.event.ConversationExecutionStartedEvent + :exclude-members: to_tracing_info + +.. _conversationexecutionfinishedevent: +.. autoclass:: wayflowcore.events.event.ConversationExecutionFinishedEvent + :exclude-members: to_tracing_info + +.. _toolexecutionstartevent: +.. autoclass:: wayflowcore.events.event.ToolExecutionStartEvent + :exclude-members: to_tracing_info + +.. _toolexecutionstreamingchunkreceivedevent: +.. autoclass:: wayflowcore.events.event.ToolExecutionStreamingChunkReceivedEvent + :exclude-members: to_tracing_info + +.. _toolexecutionresultevent: +.. autoclass:: wayflowcore.events.event.ToolExecutionResultEvent + :exclude-members: to_tracing_info + +.. _toolconfirmationrequeststartevent: +.. autoclass:: wayflowcore.events.event.ToolConfirmationRequestStartEvent + :exclude-members: to_tracing_info + +.. _toolconfirmationrequestendevent: +.. autoclass:: wayflowcore.events.event.ToolConfirmationRequestEndEvent + :exclude-members: to_tracing_info + +.. _stepinvocationstartevent: +.. autoclass:: wayflowcore.events.event.StepInvocationStartEvent + :exclude-members: to_tracing_info + +.. _stepinvocationresultevent: +.. autoclass:: wayflowcore.events.event.StepInvocationResultEvent + :exclude-members: to_tracing_info + +.. _contextproviderexecutionrequestevent: +.. autoclass:: wayflowcore.events.event.ContextProviderExecutionRequestEvent + :exclude-members: to_tracing_info + +.. _contextproviderexecutionresultevent: +.. autoclass:: wayflowcore.events.event.ContextProviderExecutionResultEvent + :exclude-members: to_tracing_info + +.. _flowexecutioniterationstartedevent: +.. autoclass:: wayflowcore.events.event.FlowExecutionIterationStartedEvent + :exclude-members: to_tracing_info + +.. _flowexecutioniterationfinishedevent: +.. autoclass:: wayflowcore.events.event.FlowExecutionIterationFinishedEvent + :exclude-members: to_tracing_info + +.. _agentexecutioniterationstartedevent: +.. autoclass:: wayflowcore.events.event.AgentExecutionIterationStartedEvent + :exclude-members: to_tracing_info + +.. _agentexecutioniterationfinishedevent: +.. autoclass:: wayflowcore.events.event.AgentExecutionIterationFinishedEvent + :exclude-members: to_tracing_info + +.. _exceptionraisedevent: +.. autoclass:: wayflowcore.events.event.ExceptionRaisedEvent + :exclude-members: to_tracing_info + +.. _agentnextactiondecisionstartevent: +.. autoclass:: wayflowcore.events.event.AgentNextActionDecisionStartEvent + :exclude-members: to_tracing_info + +.. _agentdecidednextactionevent: +.. autoclass:: wayflowcore.events.event.AgentDecidedNextActionEvent + :exclude-members: to_tracing_info + +.. seealso:: + See the table of supported events with one line description for each in the :ref:`How to Use the Event System `. + +.. _eventlisteners: + +Event Listeners +--------------- + +.. _eventlistener: +.. autoclass:: wayflowcore.events.eventlistener.EventListener + +.. _genericeventlistener: +.. autoclass:: wayflowcore.events.eventlistener.GenericEventListener + +.. _registereventlisteners: +.. autofunction:: wayflowcore.events.eventlistener.register_event_listeners + +.. _geteventlisteners: +.. autofunction:: wayflowcore.events.eventlistener.get_event_listeners + +.. _recordevent: +.. autofunction:: wayflowcore.events.eventlistener.record_event diff --git a/26.1.2/_sources/core/api/flows.rst.txt b/26.1.2/_sources/core/api/flows.rst.txt new file mode 100644 index 000000000..5881074f5 --- /dev/null +++ b/26.1.2/_sources/core/api/flows.rst.txt @@ -0,0 +1,234 @@ +Flows +===== + +This page presents all APIs and classes related to flows and steps in WayFlow. + + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/flows.html + :link-alt: Agent Spec - Flows API Reference + + Visit the Agent Spec API Documentation to learn more about Flow Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + +Flows & Steps +------------- + +.. _flow: +.. autoclass:: wayflowcore.flow.Flow + :exclude-members: execute + +.. _assistantstep: +.. autoclass:: wayflowcore.steps.step.Step + + +.. _dataflowedge: +.. autoclass:: wayflowcore.dataconnection.DataFlowEdge + +.. _controlflowedge: +.. autoclass:: wayflowcore.controlconnection.ControlFlowEdge + +.. _presentstep: + +Task steps +---------- + +.. _agentexecutionstep: +.. autoclass:: wayflowcore.steps.agentexecutionstep.AgentExecutionStep + +.. _promptexecutionstep: +.. autoclass:: wayflowcore.steps.promptexecutionstep.PromptExecutionStep + +.. _templaterenderingstep: +.. autoclass:: wayflowcore.steps.templaterenderingstep.TemplateRenderingStep + +.. _toolexecutionstep: +.. autoclass:: wayflowcore.steps.toolexecutionstep.ToolExecutionStep + +.. _extractvaluefromjsonstep: +.. autoclass:: wayflowcore.steps.textextractionstep.extractvaluefromjsonstep.ExtractValueFromJsonStep + +.. _regexextractionstep: +.. autoclass:: wayflowcore.steps.textextractionstep.regexextractionstep.RegexExtractionStep + +.. autoclass:: wayflowcore.steps.promptexecutionstep.StructuredGenerationMode +.. _searchstep: +.. autoclass:: wayflowcore.steps.searchstep.SearchStep + +.. _datastoresteps: + +Datastore tasks +~~~~~~~~~~~~~~~ + +.. admonition:: On the transactional consistency of datastore tasks + + When executing Datastore tasks in a flow, each step will execute one + atomic operation on the datastore (that is, one transaction in + database-backed datastores). Therefore, rolling-back a sequence of + operations in case one or more steps fail during execution is not + supported. Please keep this in mind when designing flows using these + steps. + +.. _datastoreliststep: +.. autoclass:: wayflowcore.steps.datastoresteps.DatastoreListStep + +.. _datastorecreatestep: +.. autoclass:: wayflowcore.steps.datastoresteps.DatastoreCreateStep + +.. _datastoreupdatestep: +.. autoclass:: wayflowcore.steps.datastoresteps.DatastoreUpdateStep + +.. _datastoredeletestep: +.. autoclass:: wayflowcore.steps.datastoresteps.DatastoreDeleteStep + +.. _datastorequerystep: +.. autoclass:: wayflowcore.steps.datastoresteps.DatastoreQueryStep + +IO steps +-------- + +.. _inputmessagestep: +.. autoclass:: wayflowcore.steps.inputmessagestep.InputMessageStep + +.. _outputmessagestep: +.. autoclass:: wayflowcore.steps.outputmessagestep.OutputMessageStep + +.. _getchathistorystep: +.. autoclass:: wayflowcore.steps.getchathistorystep.GetChatHistoryStep + +.. _variablestep: +.. autoclass:: wayflowcore.steps.variablesteps.variablestep.VariableStep + +.. _variablereadstep: +.. autoclass:: wayflowcore.steps.variablesteps.variablereadstep.VariableReadStep + +.. _variablewritestep: +.. autoclass:: wayflowcore.steps.variablesteps.variablewritestep.VariableWriteStep + +.. _constantvaluesstep: +.. autoclass:: wayflowcore.steps.ConstantValuesStep + +.. autoclass:: wayflowcore.steps.getchathistorystep.MessageSlice + +Flow steps +---------- + +.. _retrystep: +.. autoclass:: wayflowcore.steps.retrystep.RetryStep + +.. _catchexceptionstep: +.. autoclass:: wayflowcore.steps.catchexceptionstep.CatchExceptionStep + +.. _flowexecutionstep: +.. autoclass:: wayflowcore.steps.flowexecutionstep.FlowExecutionStep + +.. _parallelflowexecutionstep: +.. autoclass:: wayflowcore.steps.parallelflowexecutionstep.ParallelFlowExecutionStep + +.. _mapstep: +.. autoclass:: wayflowcore.steps.mapstep.MapStep + +.. _parallelmapstep: +.. autoclass:: wayflowcore.steps.mapstep.ParallelMapStep + +.. _branchingstep: +.. autoclass:: wayflowcore.steps.branchingstep.BranchingStep + +.. _choiceselectionstep: +.. autoclass:: wayflowcore.steps.choiceselectionstep.ChoiceSelectionStep + +.. _completestep: +.. autoclass:: wayflowcore.steps.completestep.CompleteStep + +.. _startstep: +.. autoclass:: wayflowcore.steps.startstep.StartStep + +.. _apicallstep: +.. autoclass:: wayflowcore.steps.apicallstep.ApiCallStep + +.. autoclass:: wayflowcore.stepdescription.StepDescription + +Classes for the IO system properties +------------------------------------ + +.. _property: +.. autoclass:: wayflowcore.property.Property + +.. _booleanproperty: +.. autoclass:: wayflowcore.property.BooleanProperty + +.. _floatproperty: +.. autoclass:: wayflowcore.property.FloatProperty + +.. _messageproperty: +.. autoclass:: wayflowcore.property.MessageProperty + +.. _integerproperty: +.. autoclass:: wayflowcore.property.IntegerProperty + +.. _stringproperty: +.. autoclass:: wayflowcore.property.StringProperty + +.. _anyproperty: +.. autoclass:: wayflowcore.property.AnyProperty + +.. _listproperty: +.. autoclass:: wayflowcore.property.ListProperty + +.. _dictproperty: +.. autoclass:: wayflowcore.property.DictProperty + +.. _objectproperty: +.. autoclass:: wayflowcore.property.ObjectProperty + +.. _unionproperty: +.. autoclass:: wayflowcore.property.UnionProperty + +.. _nullproperty: +.. autoclass:: wayflowcore.property.NullProperty + +.. _vectorproperty: +.. autoclass:: wayflowcore.property.VectorProperty + +.. autoclass:: wayflowcore.property._empty_default + +.. autoclass:: wayflowcore.property.JsonSchemaParam + + +Other classes and helpers used in fixed flows +--------------------------------------------- + +.. _assistantstepresult: +.. autoclass:: wayflowcore.steps.step.StepResult + +.. autoclass:: wayflowcore.steps.step.StepExecutionStatus + +.. autoclass:: wayflowcore.flowhelpers.run_step_and_return_outputs + +.. autoclass:: wayflowcore.flowhelpers.run_flow_and_return_outputs + +.. autoclass:: wayflowcore.flowhelpers.create_single_step_flow + + + +Flow Builder +------------ + +The Flow Builder provides a concise, chainable API to assemble WayFlow Flows programmatically. +It helps wire control and data edges, use conditional branching, set entry/finish points, +and serialize flows to JSON/YAML. + +See code examples in the :ref:`Reference Sheet `. + +.. _flowbuilder: +.. autoclass:: wayflowcore.flowbuilder.FlowBuilder diff --git a/26.1.2/_sources/core/api/index.rst.txt b/26.1.2/_sources/core/api/index.rst.txt new file mode 100644 index 000000000..eeb7b8f69 --- /dev/null +++ b/26.1.2/_sources/core/api/index.rst.txt @@ -0,0 +1,49 @@ +.. _api: + +API Reference +============= + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/index.html + :link-alt: Agent Spec - API Reference + + Visit the Agent Spec API Documentation to learn more about the native Agent Spec Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + + +.. toctree:: + :maxdepth: 3 + + Agent Spec Adapters + Conversations + LLMs + Events + Flows + Agents + Agent Server + Embedding Models + Retries + Context Providers + Execution Interrupts + PromptTemplates + Serialization/Deserialization + Tools + Tracing + Variables + Datastores + Auth + Search + Warnings + Evaluation + Component + A2A diff --git a/26.1.2/_sources/core/api/interrupts.rst.txt b/26.1.2/_sources/core/api/interrupts.rst.txt new file mode 100644 index 000000000..2400b64bf --- /dev/null +++ b/26.1.2/_sources/core/api/interrupts.rst.txt @@ -0,0 +1,37 @@ +Execution Interrupts +==================== + +We often need to interrupt the normal execution of an assistant to perform specific operations including but not limited to: + +- Checking the token count +- Verifying the runtime +- Doing step by step debugging + +For this reason, we provide execution interrupts, which let developers interact with the assistant's execution at specific moments. + +This page presents all APIs and classes related to execution interrupts in WayFlow. + +Execution Interrupt interface +----------------------------- + +.. _executioninterrupt: +.. autoclass:: wayflowcore.executors.interrupts.executioninterrupt.ExecutionInterrupt + +.. _interruptedexecutionstatus: +.. autoclass:: wayflowcore.executors.interrupts.executioninterrupt.InterruptedExecutionStatus + + +Basic Execution Interrupt classes +--------------------------------- + +Timeout Execution Interrupt +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _softtimeoutexecutioninterrupt: +.. autoclass:: wayflowcore.executors.interrupts.timeoutexecutioninterrupt.SoftTimeoutExecutionInterrupt + +Token Limit Execution Interrupt +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _softtokenlimitexecutioninterrupt: +.. autoclass:: wayflowcore.executors.interrupts.tokenlimitexecutioninterrupt.SoftTokenLimitExecutionInterrupt diff --git a/26.1.2/_sources/core/api/llmmodels.rst.txt b/26.1.2/_sources/core/api/llmmodels.rst.txt new file mode 100644 index 000000000..9a6805309 --- /dev/null +++ b/26.1.2/_sources/core/api/llmmodels.rst.txt @@ -0,0 +1,160 @@ +LLMs +==== + +This page presents all APIs and classes related to LLM models. + + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/llmmodels.html + :link-alt: Agent Spec - LLMs API Reference + + Visit the Agent Spec API Documentation to learn more about LLMs Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + + +LlmModel +-------- + +.. _llmmodel: +.. autoclass:: wayflowcore.models.llmmodel.LlmModel + +.. autoclass:: wayflowcore.models._requesthelpers.StreamChunkType + +LlmModelFactory +--------------- + +.. _llmmodelfactory: +.. autoclass:: wayflowcore.models.llmmodelfactory.LlmModelFactory + +LlmCompletion +------------- + +.. _llmmodelcompletion: +.. autoclass:: wayflowcore.models.LlmCompletion + +Prompt +------ + +.. _prompt: +.. autoclass:: wayflowcore.models.Prompt + + +Token Usage +----------- + +Class that is used to gather all token usage information. + +.. _tokenusage: +.. autoclass:: wayflowcore.tokenusage.TokenUsage + + +LLM Generation Config +--------------------- + +Parameters for LLM generation (``max_tokens``, ``temperature``, ``top_p``, ``top_logprobs``). + +.. _llmgenerationconfig: +.. autoclass:: wayflowcore.models.llmgenerationconfig.LlmGenerationConfig + +API Type +----------- + +Class that is used to select the OpenAI API Type to use. + +.. _openaiapitype: +.. autoclass:: wayflowcore.models.openaiapitype.OpenAIAPIType + +.. _allllms: + +All models +---------- + +OpenAI Compatible Models +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _openaicompatiblemodel: +.. autoclass:: wayflowcore.models.openaicompatiblemodel.OpenAICompatibleModel + +OpenAI Models +~~~~~~~~~~~~~ + +.. _openaimodel: +.. autoclass:: wayflowcore.models.openaimodel.OpenAIModel + +Gemini Models +~~~~~~~~~~~~~ + +.. _geminimodel: +.. autoclass:: wayflowcore.models.geminimodel.GeminiModel + +.. autoclass:: wayflowcore.models.geminimodel.GeminiApiKeyAuth + +.. autoclass:: wayflowcore.models.geminimodel.GeminiCloudAuth + +Ollama Models +~~~~~~~~~~~~~ + +.. _ollamamodel: +.. autoclass:: wayflowcore.models.ollamamodel.OllamaModel + +VLLM Models +~~~~~~~~~~~ + +.. _vllmmodel: +.. autoclass:: wayflowcore.models.vllmmodel.VllmModel + +OCI GenAI Models +~~~~~~~~~~~~~~~~ + +.. _ocigenaimodel: +.. autoclass:: wayflowcore.models.ocigenaimodel.OCIGenAIModel + +.. autoclass:: wayflowcore.models.ocigenaimodel.ServingMode + +.. autoclass:: wayflowcore.models.ocigenaimodel.ModelProvider + +.. autoclass:: wayflowcore.models.ocigenaimodel.OciAPIType + +.. _ociclientconfigclassesforauthentication: + +OCI Client Config Classes for Authentication +******************************************** + +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfig + +.. _ociclientconfigwithapikey: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfigWithApiKey + +.. _ociclientconfigwithsecuritytoken: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfigWithSecurityToken + +.. _ociclientconfigwithinstanceprincipal: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfigWithInstancePrincipal + +.. _ociclientconfigwithresourceprincipal: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfigWithResourcePrincipal + +.. _ociclientconfigwithuserauthentication: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIClientConfigWithUserAuthentication + +.. autoclass:: wayflowcore.models.ociclientconfig._OCIAuthType + +.. important:: + ``OCIClientConfigWithUserAuthentication`` supports the same authentication type as ``OCIClientConfigWithApiKey`` but without a config file. + Values in the config file are passed directly through ``OCIUserAuthenticationConfig`` below. + +.. _ociuserauthenticationconfig: +.. autoclass:: wayflowcore.models.ociclientconfig.OCIUserAuthenticationConfig + +.. important:: + The serialization of this class is currently not supported since the values are sensitive information. diff --git a/26.1.2/_sources/core/api/prompttemplate.rst.txt b/26.1.2/_sources/core/api/prompttemplate.rst.txt new file mode 100644 index 000000000..53012021c --- /dev/null +++ b/26.1.2/_sources/core/api/prompttemplate.rst.txt @@ -0,0 +1,61 @@ +PromptTemplate +============== + +This page presents all APIs and classes related to prompt Templates. + +.. _prompttemplate: +.. autoclass:: wayflowcore.templates.template.PromptTemplate + +OutputParser +------------ + +.. _outputparser: +.. autoclass:: wayflowcore.outputparser.OutputParser + +.. _regexoutputparser: +.. autoclass:: wayflowcore.outputparser.RegexOutputParser + +.. _regexpattern: +.. autoclass:: wayflowcore.outputparser.RegexPattern + +.. _jsonoutputparser: +.. autoclass:: wayflowcore.outputparser.JsonOutputParser + +.. _tooloutputparser: +.. autoclass:: wayflowcore.outputparser.ToolOutputParser + + +Message transforms +------------------ + +.. _messagetransform: +.. autoclass:: wayflowcore.transforms.MessageTransform + +.. autoclass:: wayflowcore.transforms.CoalesceSystemMessagesTransform + +.. autoclass:: wayflowcore.transforms.RemoveEmptyNonUserMessageTransform + +.. autoclass:: wayflowcore.transforms.AppendTrailingSystemMessageToUserMessageTransform + +.. autoclass:: wayflowcore.transforms.SplitPromptOnMarkerMessageTransform + +.. _canonicalizationtransform: +.. autoclass:: wayflowcore.transforms.CanonicalizationMessageTransform + +.. _messagesummarizationtransform: +.. autoclass:: wayflowcore.transforms.MessageSummarizationTransform + +.. _conversationsummarizationtransform: +.. autoclass:: wayflowcore.transforms.ConversationSummarizationTransform + +Helpers +------- + +.. _prompttemplatehelpers: +.. autofunction:: wayflowcore.templates.structuredgeneration.adapt_prompt_template_for_json_structured_generation + +.. autoclass:: wayflowcore._utils._templating_helpers.ToolRequestAsDictT + +.. autoclass:: wayflowcore._utils._templating_helpers.ToolResultAsDictT + +.. autoclass:: wayflowcore._utils._templating_helpers.MessageAsDictT diff --git a/26.1.2/_sources/core/api/retries.rst.txt b/26.1.2/_sources/core/api/retries.rst.txt new file mode 100644 index 000000000..90e6b79b4 --- /dev/null +++ b/26.1.2/_sources/core/api/retries.rst.txt @@ -0,0 +1,15 @@ +Retries +======= + +This page presents the API related to retry configuration for remote components in WayFlow. + +Retry Policy +------------ + +.. _retrypolicy: +.. autoclass:: wayflowcore.retrypolicy.RetryPolicy + +Retry Jitter +------------ + +.. autoclass:: wayflowcore.retrypolicy.RetryJitter diff --git a/26.1.2/_sources/core/api/search.rst.txt b/26.1.2/_sources/core/api/search.rst.txt new file mode 100644 index 000000000..39be77753 --- /dev/null +++ b/26.1.2/_sources/core/api/search.rst.txt @@ -0,0 +1,63 @@ +.. _search_api: + +Search +====== + +This section documents the APIs for the core Search, Vector, and related configuration/data classes. + +Search Configuration +-------------------- + +.. _search_config: +.. autoclass:: wayflowcore.search.config.SearchConfig + +.. _vector_config: +.. autoclass:: wayflowcore.search.config.VectorConfig + +.. _retriever_config: +.. autoclass:: wayflowcore.search.config.RetrieverConfig + +.. _vector_retriever_config: +.. autoclass:: wayflowcore.search.config.VectorRetrieverConfig + +.. _serializer_config: +.. autoclass:: wayflowcore.search.config.SerializerConfig + +.. _concat_serializer_config: +.. autoclass:: wayflowcore.search.config.ConcatSerializerConfig + +Similarity Metrics +------------------ + +.. _similarity_metric: +.. autoclass:: wayflowcore.search.metrics.SimilarityMetric + +Search Toolbox +-------------- + +.. _search_tool_box: +.. autoclass:: wayflowcore.search.toolbox.SearchToolBox + +Vector Generator +---------------- + +.. _vector_generator: +.. autoclass:: wayflowcore.search.vectorgenerator.VectorGenerator + +.. _simple_vector_generator: +.. autoclass:: wayflowcore.search.vectorgenerator.SimpleVectorGenerator + +Vector Indexes +-------------- + +.. _vector_index: +.. autoclass:: wayflowcore.search.vectorindex.VectorIndex + +.. _oracle_vector_index: +.. autoclass:: wayflowcore.search.vectorindex.OracleDatabaseVectorIndex + +.. _inmemory_vector_index: +.. autoclass:: wayflowcore.search.vectorindex.BaseInMemoryVectorIndex + +.. _entity_vector_index: +.. autoclass:: wayflowcore.search.vectorindex.EntityVectorIndex diff --git a/26.1.2/_sources/core/api/serialization.rst.txt b/26.1.2/_sources/core/api/serialization.rst.txt new file mode 100644 index 000000000..28d1e3c5a --- /dev/null +++ b/26.1.2/_sources/core/api/serialization.rst.txt @@ -0,0 +1,47 @@ +.. _serialization: + +Serialization / Deserialization +=============================== + +Base classes +------------ + +.. _serializableobject: +.. autoclass:: wayflowcore.serialization.serializer.SerializableObject + + +Serialization +------------- + +.. _serializationcontext: +.. autoclass:: wayflowcore.serialization.context.SerializationContext + +.. _serialize: +.. autofunction:: wayflowcore.serialization.serializer.serialize + + +Deserialization +--------------- + +.. _deserializationcontext: +.. autoclass:: wayflowcore.serialization.context.DeserializationContext + +.. _deserialize: +.. autofunction:: wayflowcore.serialization.serializer.deserialize + +.. _autodeserialize: +.. autofunction:: wayflowcore.serialization.serializer.autodeserialize + + +Plugins +------- + +WayFlow Plugins are the expected mean that users can use to introduce new concepts and components, or extensions to existing ones, +such that they can be integrated seamlessly into the serialization, deserialization, and Agent Spec conversion processes +of WayFlow. + +.. _wayflowserializationplugin: +.. autoclass:: wayflowcore.serialization.plugins.WayflowSerializationPlugin + +.. _wayflowdeserializationplugin: +.. autoclass:: wayflowcore.serialization.plugins.WayflowDeserializationPlugin diff --git a/26.1.2/_sources/core/api/tools.rst.txt b/26.1.2/_sources/core/api/tools.rst.txt new file mode 100644 index 000000000..e95d7a544 --- /dev/null +++ b/26.1.2/_sources/core/api/tools.rst.txt @@ -0,0 +1,122 @@ +Tools +===== + +This page presents all APIs and classes related to tools in WayFlow + + +.. |agentspec-icon| image:: ../../_static/icons/agentspec-icon.svg + :width: 100px + +.. grid:: 1 + + .. grid-item-card:: |agentspec-icon| + :link: https://oracle.github.io/agent-spec/api/tools.html + :link-alt: Agent Spec - Tools API Reference + + Visit the Agent Spec API Documentation to learn more about LLMs Components. + + +.. tip:: + + Click the button above ↑ to visit the `Agent Spec Documentation `_ + + +Tool +---- + +This is the base class for tools. + +.. _tool: +.. autoclass:: wayflowcore.tools.tools.Tool + + +Client Tool +----------- + +.. _clienttool: +.. autoclass:: wayflowcore.tools.clienttools.ClientTool + +Server Tool +----------- + +.. _servertool: +.. autoclass:: wayflowcore.tools.servertools.ServerTool + +Tool from ToolBox +----------------- + +.. _toolfromtoolbox: +.. autoclass:: wayflowcore.tools.toolfromtoolbox.ToolFromToolBox + +Remote Tool +----------- + +.. _remotetool: +.. autoclass:: wayflowcore.tools.remotetools.RemoteTool + +MCP Tool +-------- + +`Model Context Protocol `_ (MCP) is an open protocol that standardizes how applications provide context to LLMs. + +.. _mcptool: +.. autoclass:: wayflowcore.mcp.MCPTool + +.. _mcptoolbox: +.. autoclass:: wayflowcore.mcp.MCPToolBox + +.. _mcpstreamingtool: +.. autofunction:: wayflowcore.mcp.mcphelpers.mcp_streaming_tool + +.. _sessionparameters: +.. autoclass:: wayflowcore.mcp.SessionParameters + :exclude-members: to_dict + +.. _authlessmcpenabled: +.. autofunction:: wayflowcore.mcp.authless_mcp_enabled + +.. _enablemcpwithoutauth: +.. autofunction:: wayflowcore.mcp.enable_mcp_without_auth + +.. _mcpoauthconfigfactory: +.. autoclass:: wayflowcore.mcp.MCPOAuthConfigFactory + +.. _clienttransport: +.. autoclass:: wayflowcore.mcp.ClientTransport + +.. _stdiotransport: +.. autoclass:: wayflowcore.mcp.StdioTransport + +.. _ssetransport: +.. autoclass:: wayflowcore.mcp.SSETransport + +.. _ssemtlstransport: +.. autoclass:: wayflowcore.mcp.SSEmTLSTransport + +.. _streamablehttptransport: +.. autoclass:: wayflowcore.mcp.StreamableHTTPTransport + +.. _streamablehttpmtlstransport: +.. autoclass:: wayflowcore.mcp.StreamableHTTPmTLSTransport + + +Tool decorator +-------------- + +.. _tooldecorator: +.. autofunction:: wayflowcore.tools.toolhelpers.tool + +Tool Request and Result Classes +------------------------------- + +.. _toolrequest: +.. autoclass:: wayflowcore.tools.tools.ToolRequest + +.. _toolresult: +.. autoclass:: wayflowcore.tools.tools.ToolResult + +ToolBox +------- + +.. _toolbox: +.. autoclass:: wayflowcore.tools.toolbox.ToolBox diff --git a/26.1.2/_sources/core/api/tracing.rst.txt b/26.1.2/_sources/core/api/tracing.rst.txt new file mode 100644 index 000000000..fe9f83015 --- /dev/null +++ b/26.1.2/_sources/core/api/tracing.rst.txt @@ -0,0 +1,73 @@ +.. _tracing: + +Tracing +======= + +This page presents all APIs and classes related to tracing in WayFlow. + + +Trace +------ + +.. _get_trace: +.. autofunction:: wayflowcore.tracing.trace.get_trace + +.. _trace: +.. autoclass:: wayflowcore.tracing.trace.Trace + + +Spans +----- + +.. _get_current_span: +.. autofunction:: wayflowcore.tracing.span.get_current_span + +.. _get_active_span_stack: +.. autofunction:: wayflowcore.tracing.span.get_active_span_stack + +.. _span: +.. autoclass:: wayflowcore.tracing.span.Span + +.. _llm_generation_span: +.. autoclass:: wayflowcore.tracing.span.LlmGenerationSpan + +.. _conversational_component_execution_span: +.. autoclass:: wayflowcore.tracing.span.ConversationalComponentExecutionSpan + +.. _conversation_span: +.. autoclass:: wayflowcore.tracing.span.ConversationSpan + +.. _tool_execution_span: +.. autoclass:: wayflowcore.tracing.span.ToolExecutionSpan + +.. _step_invocation_span: +.. autoclass:: wayflowcore.tracing.span.StepInvocationSpan + +.. _context_provider_execution_span: +.. autoclass:: wayflowcore.tracing.span.ContextProviderExecutionSpan + +Span Processors +---------------- + +.. _spanprocessor: +.. autoclass:: wayflowcore.tracing.spanprocessor.SpanProcessor + +.. _simplespanprocessor: +.. autoclass:: wayflowcore.tracing.spanprocessor.SimpleSpanProcessor + + +Span Exporter +------------- + +.. _spanexporter: +.. autoclass:: wayflowcore.tracing.spanexporter.SpanExporter + + +OpenTelemetry +------------- + +.. _otelsimplespanprocessor: +.. autoclass:: wayflowcore.tracing.opentelemetry.spanprocessor.OtelSimpleSpanProcessor + +.. _otelbatchspanprocessor: +.. autoclass:: wayflowcore.tracing.opentelemetry.spanprocessor.OtelBatchSpanProcessor diff --git a/26.1.2/_sources/core/api/variables.rst.txt b/26.1.2/_sources/core/api/variables.rst.txt new file mode 100644 index 000000000..bce02fbc2 --- /dev/null +++ b/26.1.2/_sources/core/api/variables.rst.txt @@ -0,0 +1,19 @@ +Variables +========= + +This page presents all APIs and classes related to variables in WayFlow. + +.. _variable: +.. autoclass:: wayflowcore.variable.Variable + +.. _variablewriteoperation: +.. autoclass:: wayflowcore.variable.VariableWriteOperation + +.. autoclass:: wayflowcore.steps.variablesteps.variablestep.VariableStep + :noindex: + +.. autoclass:: wayflowcore.steps.variablesteps.variablereadstep.VariableReadStep + :noindex: + +.. autoclass:: wayflowcore.steps.variablesteps.variablewritestep.VariableWriteStep + :noindex: diff --git a/26.1.2/_sources/core/api/warnings.rst.txt b/26.1.2/_sources/core/api/warnings.rst.txt new file mode 100644 index 000000000..26bbfd82c --- /dev/null +++ b/26.1.2/_sources/core/api/warnings.rst.txt @@ -0,0 +1,7 @@ +Warnings +======== + +This page presents all warning classes in WayFlow. + +.. _securitywarning: +.. autoclass:: wayflowcore.warnings.SecurityWarning diff --git a/26.1.2/_sources/core/changelog.rst.txt b/26.1.2/_sources/core/changelog.rst.txt new file mode 100644 index 000000000..31dbf53ba --- /dev/null +++ b/26.1.2/_sources/core/changelog.rst.txt @@ -0,0 +1,583 @@ +Changelog +========= + +WayFlow |current_version| +------------------------- + +New features +^^^^^^^^^^^^ + +* **Configurable retry policies for remote components** + + Added the ``RetryPolicy`` object to configure retries, backoff, + and request timeouts for components doing remote calls. + + For usage details, see :doc:`the new retry configuration guide `. + +* **Custom TLS certificates for OpenAI-compatible models:** + + :ref:`OpenAICompatibleModel ` and :ref:`OpenAICompatibleEmbeddingModel ` + now support optional ``key_file``, ``cert_file`` and ``ca_file`` parameters for custom CAs and mutual TLS. + + For more information read :ref:`how to configure custom TLS certificates for OpenAI-compatible endpoints `. + +* **OAuth support for MCP Clients:** + + MCP Clients now support OAuth-based authorization. + + For more information read the :doc:`API Reference ` and the guide on + :doc:`how to connect assistants to MCP servers `. + +* **Message Summarization Transforms and Datastores can be converted between Agent Spec and Wayflow** + + Added support for converting `MessageSummarizationTransform` and `ConversationSummarizationTransform` between Agent Spec and Wayflow. Similarly for Datastores (`OracleDatabaseDatastore`, `PostgreSQLDatabaseDatastore`). + You can now declare your agents with summarization transforms and summary caching in Agent Spec and run them in WayFlow. + +* **Gemini models (Vertex AI + AI Studio):** + + Added ``GeminiModel`` to run Gemini models via Google Vertex AI and Google AI Studio. + For more information read the :doc:`API Reference on LLM models ` and the guide on + :doc:`how to use LLMs from different providers `. + + +* **Logprob support in `LlmGenerationConfig` and `PromptExecutionStep`** + + Add per-token log-probabilities support with the ``top_logprobs`` generation config parameter and support returning + per-token log-probabilities in the ``PromptExecutionStep``. + For more information please read the guide on :ref:`How to request per-token log-probabilities ` + + + +Improvements +^^^^^^^^^^^^ + +* **Improved Agent Spec tracing compatibility** + + Agent Spec tracing exports now serialize WayFlow Agent Spec plugin components with the + proper plugin context and report a valid flow end branch when a flow finishes through a + transition to ``None``. + +* **Scoped opt-in for authless MCP clients** + + Added ``authless_mcp_enabled()`` as a scoped context manager for local or test MCP clients + that intentionally do not use authentication. The existing ``enable_mcp_without_auth()`` + helper remains available, but now emits a ``SecurityWarning``. Authless MCP use also emits + a ``SecurityWarning`` when validation is bypassed. + +* **Improved TLS defaults for MCP HTTP transports** + + ``SSETransport`` and ``StreamableHTTPTransport`` now use standard HTTPS certificate + verification by default. ``SSEmTLSTransport`` and ``StreamableHTTPmTLSTransport`` + now also verify that the server certificate matches the requested hostname unless + this check is explicitly disabled. + +* **Improved agent server access defaults** + + ``A2AServer.run(api_key=...)`` now enforces bearer-token authentication when an API key is + provided. ``A2AServer`` and ``OpenAIResponsesServer`` now require an API key when binding to + non-loopback hosts, while unauthenticated loopback development remains available with a warning. + ``OpenAIResponsesServer`` no longer enables CORS by default and supports explicit allowed + origins configuration. + +* **Agent Spec component loading policies can be customized** + + ``AgentSpecLoader`` now accepts ``allowed_components`` and ``blocked_components`` + so applications can control which Agent Spec and WayFlow plugin component types + are allowed to load. This is useful for trusted deployments that need to opt in + to runtime-sensitive components, and for stricter deployments that want to + explicitly allow only a known component set. WayFlow extends PyAgentSpec's default + blocked components with WayFlow-specific runtime-sensitive components. + +* **Removed ``starlette`` from third party dependencies** + + Direct ``starlette`` imports were removed in favor of FastAPI equivalent alternatives, so ``wayflowcore`` no longer + relies on ``starlette`` as a direct, third party dependency. + +* **Improved tool-result formatting across templates and model adapters** + + Built-in non-native prompt templates and model adapters now render tool results + through a shared JSON-based formatter, which keeps tool-returned data more + consistent across providers. + +Documentation +^^^^^^^^^^^^^ + +* **Added a new MCP server to browse the WayFlow documentation:** + + Added a new MCP server in ``examples/mcp/docs_mcp.py`` to let assistants + browse the WayFlow documentation. + + For more information, see the :doc:`Docs MCP Server guide `. + + +* **OpenAI-compatible embeddings now support API keys:** + + ``OpenAICompatibleEmbeddingModel`` now accepts an optional ``api_key`` (with ``OPENAI_API_KEY`` environment fallback) and sends it as an OpenAI-style ``Authorization: Bearer ...`` header. + +Possibly Breaking Changes +^^^^^^^^^^^^^^^^^^^^^^^^^ + +* **Authless MCP usage now requires explicit scoped opt-in** + + Unauthenticated MCP tools and toolboxes must now be constructed or loaded from + Agent Spec inside ``authless_mcp_enabled()`` unless the transport provides + authentication. The legacy ``enable_mcp_without_auth()`` helper remains + available, but emits a ``SecurityWarning``. Code that treats warnings as errors + may need to catch or filter this warning for local tests. + +* **MCP HTTPS transports now expect valid certificate configuration** + + ``SSETransport`` and ``StreamableHTTPTransport`` now use standard HTTPS certificate + verification by default, and ``SSEmTLSTransport`` and ``StreamableHTTPmTLSTransport`` + now verify the requested hostname by default. Deployments using self-signed certificates, + custom certificate authorities, or certificates whose hostname does not match the MCP + server URL may require certificate updates or transport configuration changes. + +* **Agent servers require API keys on non-loopback hosts** + + ``A2AServer`` and ``OpenAIResponsesServer`` now require an ``api_key`` when + binding to non-loopback hosts such as ``0.0.0.0`` or a public interface. + A loopback host only accepts connections from the same machine, for example + ``127.0.0.1`` or ``localhost``. Unauthenticated local development on loopback + hosts remains available with a warning. + +* **MCP stdio transports are blocked by default in Agent Spec loaders** + + Configurations containing Agent Spec ``StdioTransport`` or WayFlow ``PluginStdioTransport`` + will no longer load by default through ``AgentSpecLoader``. WayFlow inherits the + PyAgentSpec default blocked components and adds ``PluginStdioTransport``. If a trusted + configuration intentionally uses stdio transports, pass ``blocked_components=[]`` to the + loader, or provide a custom ``blocked_components`` value that does not include those + transport component types. + +* **Summarization Transforms now do not do caching if the datastore is None** + + Previously when the `datastore` arguments of `MessageSummarizationTransform` and `ConversationSummarizationTransform` were set to `None`, an in-memory datastore was automatically created. Now, the in-memory datastore is created + only if the `datastore` is left unspecified. If it is set to `None` then no caching happens. + +* **VariableStep:** + + Introducing ``VariableStep``, a comprehensive step for reading from and writing on variables. + This feature provides users with an easier-to-use API to use ``Variable`` in flows, that may be used for centralizing the state and simplify the data flows. + Especially, this may significantly simplify flows currently using the ``VariableReadStep`` and ``VariableWriteStep``. + + For more information check out :doc:`how to use variables for shared state in flows `. + +* **Deprecated `json_body` parameter in RemoteTool/ApiCallStep:** + + Deprecated the `json_body` parameter in RemoteTool/ApiCallStep in favour of the `data` parameter. A DeprecationWarning will be raised whenever + the `json_body` is tried to be set, and this parameter would be removed in version 26.2.0. If `json_body` is tried to be set, it will automatically be converted + to `data` in RemoteTool/ApiCallStep. To migrate, simply switch the `json_body` keyword argument with `data`. `data` supports both JSON-objects and form-data for + HTTP requests. + + +Bug fixes +^^^^^^^^^ + +* Fix: OpenAI-compatible Responses API requests now replay assistant text messages using + the easy-input shape accepted by OpenAI and vLLM, avoiding validation errors when vLLM + parses assistant history as output-message content. No-argument tools also emit an + explicit object parameters schema for providers that reject empty tool parameter schemas. + +* Fix: agents now handle tool calls more robustly with OpenAI-compatible and Llama models, + ignoring ``null`` streamed tool-call delta fields, normalizing model-produced arguments + against declared tool schemas, and preserving non-string Llama tool-result values. + +* Fix: ``OCIGenAIModel`` now forwards ``LlmGenerationConfig`` parameters when using + ``OciAPIType.OPENAI_CHAT_COMPLETIONS`` and ``OciAPIType.OPENAI_RESPONSES``. + +* Fix: `CanonicalizationMessageTransform` now correctly handles assistant-role tool-result messages from runtime conversation history, which fixes parallel tool-calling turns for native tool-calling prompts such as OCI Google models. + +* **Fix summarization transform persistence and multi-agent integration** + + ``MessageSummarizationTransform`` and ``ConversationSummarizationTransform`` now serialize the configuration required to deserialize + prompt templates and resume serialized conversations correctly. ``Swarm`` and ``ManagerWorkers`` also now apply their component-level + ``transforms`` during execution, making it possible to attach summarization transforms directly to these multi-agent components. + + See the guide on :ref:`using Swarm ` and :ref:`ManagerWorkers ` to learn + more about how component-level transforms are combined with prompt template transforms during execution. + +* Fix: conversations no longer replay already-materialized tool results when resuming manager-worker executions, + which could previously duplicate internal `send_message` results and break continued execution after serialization. + +WayFlow 26.1.1 +-------------- + +New features +^^^^^^^^^^^^ + +* **Tool output streaming:** + + Added a new event ``ToolExecutionStreamingChunkReceived`` to surface streamed tool chunks during tool execution. + Tool output streaming is supported for MCP tools and for ``ServerTool`` with async-generator callables. + + Check the guide on :ref:`How to enable tool output streaming ` for more information. + + +Bug fixes +^^^^^^^^^ + +* Fix: MCP tools now support union/optional output schemas for the `result` field (e.g., `anyOf` including `null`). This prevents KeyError during schema parsing. (#70) +* Fix: Gemini models with the OpenAiCompatible model now support native tool calling (#77) +* Fix: MCP session persistence no longer collides when multiple MCP client transports (different servers/connections) are used within the same conversation. +* Fix: Agent no longer ignores tool calls when the LLM attempts to both invoke a tool and respond directly to the user. + + +WayFlow 26.1.0 +-------------- + +Security +^^^^^^^^ + +* **Stricter environment for jinja templates rendering:** + + We now use a stricter version of the SandboxedEnvironment for rendering jinja templates. + No access to object attributes is allowed, only key-based access to python dictionaries, lists, and main jinja LoopContext properties are allowed. + + Check the guide on :ref:`How to write secure prompts with Jinja templating ` for more information. + +New features +^^^^^^^^^^^^ + +* **Added Flow Builder to simplify programmatic creation of WayFlow Flows.** + + The Flow Builder is a new chainable API to create WayFlow Flows more easily. + + For more information, see the :doc:`API Reference ` and the :ref:`Reference Sheet `. + +* **Added support for OCI responses API:** + + Added support for the new OCI responses API service. You can simply configure this on the `OciGenAIModel`: + + .. code-block:: python + + from wayflowcore.models import OciGenAIModel, OciAPIType + + llm = OciGenAIModel( + ..., + api_type=OciAPIType.OPENAI_RESPONSES, # to use the responses API service + ) + + + +* **Added User Confirmation for Tools in ToolBox:** + + Introduced a `requires_confirmation` flag to the base ToolBox Class. When enabled, this flag will pause tool execution of any of this toolbox's tools and emit a `ToolExecutionConfirmationStatus`, requiring explicit user confirmation before proceeding. + During confirmation, users may edit the tool’s arguments or provide a rejection reason. The tool executes only after confirmation is granted. This flag helps to let users enforce + confirmation for the entire toolbox without having to set ``requires_confirmation=True`` on every tool it provides. + +* **Agent Spec Tracing support:** + + Open Agent Specification Tracing (short: Agent Spec Tracing) is an extension of + Agent Spec that standardizes how agent and flow executions emit traces. + Wayflow now supports the emission of traces according to the Agent Spec Tracing standard. + + For more information read the guide on :doc:`How to Enable Tracing in WayFlow `. + +* **WayFlow Plugins:** + + Wayflow plugins allow users extending existing components (like Tools, Steps, etc.), or even creating + new ones, and seamlessly integrate them in WayFlow and Agent Spec serialization and deserialization. For more information read the guide on :doc:`how to use WayFlow plugins `. + +* **OpenAI Responses API Support:** + + :ref:`OpenAICompatibleModel` now supports OpenAI Responses API, which can be configured + using the ``api_type`` parameter, which accepts values from :ref:`OpenAIAPIType`. + + This enhancement allows recent OpenAI models to better leverage advanced reasoning capabilities, resulting in significant performance improvements for Wayflow-powered workflows. + + For more information check out :doc:`the how-to guide on LLMs from different providers `. + +* **Serve agents via OpenAI Responses API:** + + Added :ref:`OpenAIResponsesServer ` and + :ref:`ServerStorageConfig ` to host WayFlow agents behind OpenAI + Responses-compatible endpoints, with optional persistence through supported datastores. + + See :doc:`the new how-to guide on serving agents ` for in-memory + setup, datastore persistence, and Agent Spec export/reload examples. + +* **Serve agents via A2A protocol:** + Introduced the :ref:`A2AServer ` class for serving WayFlow conversational components using the A2A protocol. + + For more information check out :doc:`how to serve assistants with A2A protocol `. + +Improvements +^^^^^^^^^^^^ + +* **Added ``sensitive_headers`` in components that perform remote calls:** + + ``ApiCallStep``, ``RemoteTool``, and MCP ``RemoteBaseTransport`` now have a new attribute ``sensitive_headers``. + This new attribute should be used for headers that might contain sensitive information (e.g., authentication). + Its value is merged with ``headers`` when the remote call is performed, and its value is never serialized. + +* **Support list, dict and tuple output types in MCP tools:** + + MCP tools now support non-string output types (list of string, dictionary of strings + and tuple of these types) + + For more information see the guide on :doc:`using MCP tools `. + +* **Python 3.14 support** + + Introduced support for python version 3.14. + +* **Connection persistence for MCP servers:** + + MCP client sessions are now reused which improves MCP calls latency and enables + uses where maintaining a session is required. + + This does not require any change to existing assistants or execution loops. + +* **Deserialization:** + + Deserialization of large conversations with many agents is now much faster due to optimizations in + the deserialization code. + +* **Long Message Summarization including Image Support** + + Added `MessageSummarizationTransform` which summarizes messages (including images) exceeding a configurable length + threshold using a specified LLM and caches the summaries in a user-provided `DataStore` with + configurable size, LRU eviction, and entry lifetime. + +* **Long Conversation Summarization including Image Support** + + Added `ConversationSummarizationTransform` which summarizes conversations exceeding a configurable number of messages + using a specified LLM and caches the summaries in a user-provided `DataStore` with configurable size, LRU eviction, + and entry lifetime. This helps manage long conversation contexts by summarizing older parts while preserving recent + messages. + + Transforms can now be assigned to an agent through its constructor. (This is not supported in `agentspec`, so attempting to convert agents with transforms to `agentspec` will raise a `NotImplementedError`) + +* **Improve Swarm prompt template, introduce HandoffMode and support multiple tool calls in Swarm:** + + Removed redundant agent descriptions from the Swarm template and added a guidance rule that encourages agents to hand off when appropriate. + Introduced ``HandoffMode`` to Swarm. In addition to the existing modes (``True`` → ``HandoffMode.OPTIONAL``, ``False`` → ``HandoffMode.NEVER``), + a new mode ``HandoffMode.ALWAYS`` is now supported, requiring agents to always use the handoff + mechanism when delegating tasks to other agents. Read more at :ref:`HandoffMode `. + Added support for multiple tool calls in Swarm. The LLM may now emit multiple tool calls in a single response. These tools are executed sequentially, and all tool results are returned together. + + This significantly reduces token usage and improves execution speed. + +* **Support multiple tool calls for the manager agent in ManagerWorkers** + + Added support for multiple tool calls for the manager agent in ``ManagerWorkers``. The LLM may now emit multiple tool calls in a single response. These tools are executed sequentially, and all tool results are returned together. + +* **Support Swarm in Flows and ManagerWorkers:** + + Swarms and ManagerWorkers can be used in ``AgentExecutionStep`` and integrated into Flows. ``caller_input_mode`` is added to Swarms and ManagerWorkers, similar to Agents. + +* **Support for multi level ManagerWorkers** + + Added support for multiple levels of ManagerWorkers where a worker in ``ManagerWorkers`` can itself be ``ManagerWorkers`` enabling the creation of hierarchical agent structures. + +* **Added support for Gemini reasoning models (via OpenAI compatible API)** + + The ``extra_content`` returned by Gemini's OpenAI-compatible APIs is now tracked + as part of messages and tool_requests. + + +Possibly Breaking Changes +^^^^^^^^^^^^^^^^^^^^^^^^^ + +* **Stricter environment for jinja templates rendering:** + + The new version of the Environment used to render templates could raise SecurityErrors on templates previously accepted. + +* **Removed deprecated Agent/Flow.execute:** + + Removed the deprecated method of ``Agent/Flow.execute(conversation)`` in favor of ``conversation.execute()``. + +* **Removed deprecated ChatHistoryContextProvider:** + + Removed the deprecated ``ChatHistoryContextProvider`` in favor of ``FlowContextProvider`` with a ``Flow`` with ``GetChatHistoryStep``. + +* **Removed deprecated begin_step_name parameter of Flow:** + + Removed the deprecated parameter ``begin_step_name`` of the ``Flow`` class. Please use ``begin_step``, passing in a ``Step`` object instead. + +* **Enforce valid tool names:** + + When creating any type of ``Tool``, its name field must not contain whitespaces or special characters, so that LLM APIs do not return errors. See bug fix below. + + - For now, a deprecation warning is raised; in the next cycle, an error will be thrown. + +* **Deprecated 'append' parameter in PromptTemplate transform methods:** + + The ``append`` parameter in ``with_additional_post_rendering_transform`` and ``with_additional_pre_rendering_transform`` methods is deprecated. + Use ``append_last`` instead. The ``append`` parameter will be removed in WayFlow 26.3. + + +Bug fixes +^^^^^^^^^ + +* **Fixed several issues related to event tracing serialization when containing execution state:** + + Fixed an issue where ``FlowExecutionIterationStartedEvent`` and + ``FlowExecutionIterationFinishedEvent`` could raise when the execution state contained values that + are not supported by the serializer. The tracing helpers now fall back to stringifying those + values while preserving container structures. + +* **Recording of end span event in case of exception:** + + Fixed a bug where if an exception happened during a span, it would not be recorded and the span closing would + raise an unwanted warning. Now properly records the exception as an ExceptionRaisedEvent and does not throw + a warning. + +* **Default values in agents inputs were ignored:** + + Fixed a bug where if agents had input descriptors with default values set, these defaults were ignored and not + used when starting a conversation. Now default values of input descriptors are used if they are set, and no + input entry with the descriptor name is passed to the ``start_conversation`` method. + +* **Default values in tools outputs were ignored:** + + Fixed a bug where if tools had multiple output descriptors with default values set for some of them, these defaults were ignored and not + set in the tool result if the tool execution did not produce a value for them. Tools that have a single output still ignore + its default, as a return value is always assumed to be produced by the tool (possibly ``None``). + +* **Fixed warnings raised when LLM streaming generator was not properly closed:** + + Fixed a bug where streaming LLM generation in a ``chainlit`` app could raise warnings due to a non-closed generator. The generator is now properly + closed and we silence the known issue on the ``httpx`` library. + +* **Continuing an agent conversation after an exception was raised could cause an exception:** + + Fixed a bug where if an exception would occur during a tool, a sub-agent or a sub-flow call of an agent, the conversation + could not be resumed afterwards because the conversation would miss the tool results of any calls that should have + been done after the call that raised. It now posts results mentioning the call was skipped due to a previous failure. + +* **Some Agent and Flow names could cause issues when used in multi-agent patterns:** + + Fixed a bug where Agents or Flows, whose name contains whitespaces or special characters, would crash upon sending a request to the LLM provider + + - The cause was that internally, WayFlow converted a subagent of an Agent into a Tool, which is then converted to a JSON payload to submit to the LLM provider, which then returns an HTTP error if the tool's name does not match a regex. + - Now, users may specify arbitrary names for Agents and Flows. Internally, when creating Tools out of subagents and subflows, their names would be sanitized. + +* **Configuring Agents with PromptTemplates no longer requires custom instruction** + + Fixed a bug where instantiating an Agent with an ``agent_template``, ``initial_message=None`` and ``custom_instruction=None`` would raise an exception. + Now, users can fully specify the agent template without having to additionally specify initial messages or custom instructions. + +* **Hand off in Swarms can only be done by the main agent in Optional Handoff Mode** + + Fixed a bug where handoff in Swarms was not restricted properly. The ability to handoff has been removed for agents that are sent ``send_message`` in Swarms, ensuring that only the main agent can perform handoffs in ``HandoffMode.OPTIONAL`` mode. + +* **Fixed tool calling for OCI models** + + Resolved an issue that sent incorrectly formatted chat histories to OCI endpoints, degrading + tool-calling and agentic performance for ``google`` and ``cohere`` models. + All OCI models can now more reliably and efficiently leverage tool calling. + +* **Fixed agent termination when ``caller_input_mode`` was set to never** + + Corrected an issue where agents without outputs and ``caller_input_mode=CallerInputMode.NEVER`` + would repeatedly iterate until the maximum limit. + Agents can now better complete the task and exit as expected. + +* **Structured generation with nested ObjectProperties** + + Fixed a bug that was raising an error on structured generation of nested ObjectProperties with OpenAI-compatible Models. + +WayFlow 25.4.2 +-------------- + +New features +^^^^^^^^^^^^ + +* **Agent Spec structured generation:** + Open Agent Specification introduced Structured Generation in version 25.4.2. + Support for this new Agent Spec feature was added in converters. + + For more information check out :doc:`the how-to guide on Structured Generation ` + +* **Added Tool Confirmation before Execution:** + Introduced a `requires_confirmation` flag to the base Tool Class. When enabled, this flag will pause tool execution and emit a `ToolExecutionConfirmationStatus`, requiring explicit user confirmation before proceeding. + During confirmation, users may edit the tool’s arguments or provide a rejection reason. The tool executes only after confirmation is granted. + + For more information check out :doc:`the corresponding how-to guide ` + +* **Added SplitPromptOnMarkerMessageTransform:** + Introduced a new Message Transform specialization that splits prompts on a marker into multiple messages with the same role. + + We thank @richl9 for the contribution! + +Bug fixes +^^^^^^^^^ + +* **Flow input and output descriptors** + + Fixed a bug where Flow input and output descriptors were sometimes ignored: + + - All the inputs required by the steps that compose the Flow were used instead of the input descriptors provided. + - The intersection of all the outputs generated by any branch in the Flow was used instead of the output descriptors provided. + + The behavior is now: + + - Input descriptors can now be a subset of all the inputs required by the steps that compose the Flow, + as long as the missing step inputs have a default value. + - Output descriptors can be a subset of the intersection of all the outputs generated by any branch in the Flow. + This is now correctly reflected also in other parts of the package. + +* **Datastore validation** + + Fixed a bug which might cause `OracleDatabaseDatastore` to raise an exception due to concurrent changes and unsupported + data types on unrelated parts of the schema, i.e., tables and columns that are not included in the datastore schema itself. + +Miscellaneous +^^^^^^^^^^^^^ + +* **Dependency Security Updates:** + Upgraded **MCP** to **1.17.0** and **PyYAML** to **6.0.3** to resolve known security vulnerabilities, including + `GHSA-j975-95f5-7wqh `_, + `GHSA-3qhf-m339-9g5v `_, + `GHSA-6757-jp84-gxfx `_, + and `GHSA-8q59-q68h-6hv4 `_. + +Improvements +^^^^^^^^^^^^ + +* **Use execution statuses to interact with the components** + Users can now directly see the conversation state and interact with the agent via the execution status. + + .. code-block:: python + + agent = Agent(...) + tools_dict = {...} + + conversation: Conversation = agent.start_conversation() + + while True: + status = conversation.execute() + + if isintance(status, UserMessageRequestStatus): + print('Agent >> ', status.message.content) + user_response = input('User >> ') + status.submit_user_response(user_response) + elif isinstance(status, ToolRequestStatus): + for tool_request in status.tool_requests: + tool_result = ToolResult( + tool_id=tool_request.tool_request_id, + content=tools_dict[tool_request.name](**tool_request.args) + ) + status.submit_tool_result(tool_result) + elif isinstance(status, FinishedStatus): + break + + For more information check out :doc:`Reference Sheet ` + + +WayFlow 25.4.1 — Initial release +-------------------------------- + +**WayFlow is here:** Build advanced AI-powered assistants with ease! + +With this release, WayFlow provides all you need for building AI-powered assistants, supporting structured workflows, +autonomous agents, multi-agent collaboration, human-in-the-loop capabilities, and tool-based extensibility. +Modular design ensures you can rapidly build, iterate, and customize both simple and complex assistants for any task. + +Explore further: + +- :doc:`How-to Guides ` +- :doc:`Tutorials ` +- :doc:`API Reference ` diff --git a/26.1.2/_sources/core/conceptual_guides/data_flow_edges.rst.txt b/26.1.2/_sources/core/conceptual_guides/data_flow_edges.rst.txt new file mode 100644 index 000000000..a1e38f8a8 --- /dev/null +++ b/26.1.2/_sources/core/conceptual_guides/data_flow_edges.rst.txt @@ -0,0 +1,169 @@ +.. _conceptual_dataflowedges: + +===================================================== +Data Flow Edges: What are they, when are they needed? +===================================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/conceptual_dataflowedges.py + :link-alt: Data Flow Edges how-to script + + Python script/notebook for this guide. + + + +WayFlow enables the creation of different types of AI assistants including :ref:`Flows `, +which are great to use when you want to solve complex tasks with an orchestrated sequence of operations. + +Building a Flow requires you to define a few different components, including the steps that make up the Flow, +the :ref:`Control Flow edges ` between those steps (i.e., in which order should the steps be executed), +and finally the :ref:`Data Flow Edges ` that define how data moves through the Flow. + + +Basics of Data Flow Edges +========================= + + +Data Flow Edges have one main purpose: defining how data outputs from one step are passed as inputs to another step. + +In some simple Flows, WayFlow can automatically infer how data should flow between steps. +However, in more complex scenarios, you may need to explicitly define Data Flow Edges to ensure that data is routed correctly. + +Here is a simple illustration to explain this concept: + +.. image:: ../_static/conceptual/dataflowedges_basics.jpg + :align: center + :scale: 20% + :alt: How Data Flow Edges work, when they are needed + + +We have three examples: + +1. In the first Flow (at the top), the output from Step 1 is named the same way as the input of the Step 2 (A). + Therefore, WayFlow can automatically infer that the output from Step 1 should be passed as input to Step 2. +2. In the second Flow (in the middle), Step 2's input is named differently (B) than Step 1's output (A). + In this case, WayFlow cannot infer how data should flow between the two steps, and you need to explicitly + define a Data Flow Edge to connect output A from Step 1 to input B of Step 2. +3. In the third Flow (at the bottom), both Step 1 and Step 2 expose an output named (A). WayFlow cannot infer + which output should be passed to Step 3. Therefore, you need to define two Data Flow Edges: one connecting + output A from Step 1 to Step 3, and another connecting output A from Step 2 to Step 3. + + + +Data Flow Edges in Practice +=========================== + + +Now let's see more concrete examples of how you can use Data Flow Edges in your Flows. + +Example 1: Data routing with multiple outputs +--------------------------------------------- + +.. image:: ../_static/conceptual/dataflowedges_routing.jpg + :align: center + :scale: 27% + :alt: Data Flow Edges for a Flow with data routing + + +In this first example, Step 1 produces an output that needs to be sent to two different steps: Step 2 and Step 3. + +* When the name of the value is different between the steps (e.g., "A" and "B"), WayFlow cannot automatically + infer the data routing, and you need to define Data Flow Edges explicitly. +* When the name of the value is shared between the steps (e.g. "A" in the middle Flow), WayFlow can automatically + infer the data routing without the need to define Data Flow Edges. + + +.. tip:: + + To improve the readability of your Flow definitions, it is recommended to always define Data Flow Edges explicitly, + even when WayFlow can infer them automatically. + + +.. tabs:: + + .. tab:: Code with automatic data routing + + .. literalinclude:: ../code_examples/conceptual_dataflowedges.py + :language: python + :start-after: .. start-##_Flow_with_multi_output_routing + :end-before: .. end-##_Flow_with_multi_output_routing + + .. tab:: Code with explicit data routing + + .. literalinclude:: ../code_examples/conceptual_dataflowedges.py + :language: python + :start-after: .. start-##_Flow_with_multi_output_routing_with_explicit_edges + :end-before: .. end-##_Flow_with_multi_output_routing_with_explicit_edges + + + +Example 2: Looping Flows +------------------------ + + +.. image:: ../_static/conceptual/dataflowedges_looping.jpg + :align: center + :scale: 30% + :alt: Data Flow Edges for Looping Flows + +In this second example, we have a Flow that includes a loop. + +Similar to the previous example: + +* When the names of the values differ between the steps (example flow at the top), WayFlow cannot automatically + infer the data routing, and you need to define Data Flow Edges explicitly. +* When the names of the values are shared between the steps (middle example), WayFlow can automatically + infer the data routing without the need to define Data Flow Edges. + +When creating looping Flows, it is generally recommended to define Data Flow Edges explicitly to avoid confusion. + + +.. tabs:: + + .. tab:: Code with automatic data routing + + .. literalinclude:: ../code_examples/conceptual_dataflowedges.py + :language: python + :start-after: .. start-##_Flow_with_looping + :end-before: .. end-##_Flow_with_looping + + .. tab:: Code with explicit data routing + + .. literalinclude:: ../code_examples/conceptual_dataflowedges.py + :language: python + :start-after: .. start-##_Flow_with_looping_with_explicit_edges + :end-before: .. end-##_Flow_with_looping_with_explicit_edges + + + +.. hint:: + + You may have noticed that in the code for this looping flow, the input property **A** of **Step 1** has a default value. + This is required because in the first loop iteration, the flow has yet to produce a value for **A**. Later in the execution + of the flow, the **Step 2** produces a value for **A** which is then consumed by the **Step 1**. + + +Next steps +========== + +Having learned about Data Flow Edges, you may now proceed to: + +- :doc:`How to use Agents inside Flows <../howtoguides/howto_agents_in_flows>` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the +full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/conceptual_dataflowedges.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/conduct.rst.txt b/26.1.2/_sources/core/conduct.rst.txt new file mode 100644 index 000000000..618c0757f --- /dev/null +++ b/26.1.2/_sources/core/conduct.rst.txt @@ -0,0 +1,59 @@ +:orphan: + +=============== +Code of Conduct +=============== + +Pledge +------ + +In the interest of fostering an open and welcoming environment, we, as contributors and maintainers, +pledge to making participation in the project a harassment-free experience for everyone, +regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, +nationality, personal appearance, race, religion, or sexual identity and orientation. + +Standards +--------- + +Examples of behavior that contributes to creating a positive environment include: + +- Using welcoming and inclusive language +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members + +Examples of unacceptable behavior by participants include: + +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others private information, such as a physical or electronic address, without explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting + +Responsibilities +---------------- + +Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and +fair corrective action in response to any instances of unacceptable behavior. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, +or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. + +Scope +----- + +This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project. +Examples of representing a project include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. + +Enforcement +----------- + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team. +All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. +The project team is obligated to maintain confidentiality with regard to the reporter of an incident. + +Attribution +----------- + +This Code of Conduct is adapted from the Contributor Covenant, version +1.4, available `here `_. diff --git a/26.1.2/_sources/core/contributing.rst.txt b/26.1.2/_sources/core/contributing.rst.txt new file mode 100644 index 000000000..d5af139e2 --- /dev/null +++ b/26.1.2/_sources/core/contributing.rst.txt @@ -0,0 +1,113 @@ +================ +For Contributors +================ + +WayFlow is an open-source project from Oracle, and developers from around the world are welcome to contribute. + +There are several ways to collaborate: + +- By submitting a `GitHub issue `_ for bug reports or questions. +- By submitting a Request for Comments (RFC) for a new feature request or enhancement. +- By submitting a `GitHub pull request `_. + +As a contributor, we expect you to abide by the WayFlow :doc:`Contributor Code of Conduct `, which outlines the standards for respectful and constructive collaboration. + + +Submitting a GitHub Issue +------------------------- + +Use GitHub's issue tracking system to report problems (bugs) or ask questions related to the project. +You can submit a GitHub issue for WayFlow `here `_. + +When submitting a bug, provide a clear description of the issue. +We encourage you to: + +- Include steps to reproduce the bug, so project developers can replicate the problem. +- Attach error messages, logs, or screenshots to give more context to the issue. +- Mention the environment (operating system, version, etc.) where the bug occurs. + + +Submitting a Request for Comments (RFC) +--------------------------------------- + +To propose a new feature or enhancement, submit a Request for Comments (RFC). +This RFC is basically a design proposal where you can share a detailed description of what change you want to make, +why it is needed, and how you propose to implement it. +The RFC gives core maintainers an opportunity to suggest refinements before you start coding. + +Follow these instructions to submit an RFC. + +I. Create an RFC +~~~~~~~~~~~~~~~~ + +Fork the `WayFlow repository `_. +Fill out your proposal using the :doc:`provided template `. +Rename the template file to **RFC-your-feature-name.rst** and push it to your fork. +Submit a pull request titled **RFC-your-feature-name**. +Before your RFC is ready for review, give it the draft label. + +II. Get Feedback on the RFC +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once your RFC ready for review, remove the draft label. +File a GitHub issue against the `WayFlow repository `_ with request to review your proposal. +In the description, include a short summary of your feature and a link to your RFC pull request. + +The core developers will review your PR RFC and offer feedback. +Revise your proposal as needed until everyone agrees on a path forward. + +III. Implement Your Proposal +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If your RFC pull request is accepted, you can begin working on the implementation. +The next step is to submit a pull request (PR) with your code changes. +This takes the form of a regular PR review. +Be sure to link your PR to the accepted RFC so reviewers can easily catch up on the context and design decisions behind your proposal. + + +Submitting a Pull Request +------------------------- + +For smaller changes, such as bug fixes, you can proceed directly and `create a pull request (PR) `_. +This process is similar for implementing your RFC, except in this case, the PR will include the code changes. + +I. Create a Pull Request +~~~~~~~~~~~~~~~~~~~~~~~~ + +The common process is forking the `WayFlow repository `_, pushing a change, and creating a PR. +When creating a PR, make sure to include a clear description of the intention of the change. +Describe why (1) the change is needed, (2) how it is implemented, and, optionally, (3) what further implications it may have. +You can either use the PR request description field or the commit message. +It is recommended to address one fix or feature per PR request. + +Once you have `created a pull +request `_, +the CI service will run some sanity checks on your change. +Be sure to address any obvious issues caught by these checks (for example, formatting violation). + +II. Sign the Oracle Contributor Agreement +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To allow your pull request to be accepted, you need to sign the `Oracle Contributor Agreement (OCA) `_. +Sign it online, and once your name appears on the OCA signatory list, your pull request will be authorized. +If you signed the agreement, but the bot leaves a message that you have not signed the OCA, leave a comment on the pull request. +If it appears to be a delay, please send an email to *oracle-ca_us@oracle.com*. + +III. Review and Merge +~~~~~~~~~~~~~~~~~~~~~ + +An Oracle employee will review the proposed change and, once it is in a mergeable state, will take responsibility for merging it into the main branch. + + +Contributing to Documentation +----------------------------- + +The WayFlow documentation is open source and anyone can contribute to make it perfect and comprehensive. +If you consider contributing to the documentation, read our :doc:`writing guidelines ` beforehand to ensure consistency and quality. + +The end +------- + +WayFlow welcomes contributions from both users and developers! + +The WayFlow team diff --git a/26.1.2/_sources/core/faqs.rst.txt b/26.1.2/_sources/core/faqs.rst.txt new file mode 100644 index 000000000..6eeddbc9c --- /dev/null +++ b/26.1.2/_sources/core/faqs.rst.txt @@ -0,0 +1,40 @@ +Frequently Asked Questions +========================== + +**What types of assistants can I create using WayFlow?** + + You can create two main types of assistants: :ref:`Agents ` and :ref:`Flows `. Agents are conversational assistants that can + perform tasks and ask follow-up questions, while Flows are workflow-based assistants that can be represented as a flow of steps. + +**What is the main difference between Agents and Flows?** + + Agents are more autonomous but less reliable and harder to run in production, while Flows are more predictable and easier to debug. + +**How do I serialize or deserialize my assistant?** + + You can use the APIs provided by WayFlow to export/load your assistants to/from Agent Spec. You can use a few lines of code to export + your assistant and load it in WayFlow using a JSON file. See the :doc:`API reference ` for more information. + +**What common steps are available to build Flows in WayFlow?** + + The central step in WayFlow is the :ref:`prompt execution step `, which allows you to generate prompts with an LLM. + Other steps include :ref:`regex extraction `, :ref:`extract from JSON `, + :ref:`branching `, :ref:`user input `, :ref:`output `, + :ref:`flow execution `, :ref:`MapStep `, :ref:`ApiCallStep `, and others. + +**What models are available?** + + All WayFlow LLM models have the same API, but they are powered by different models underneath. WayFlow currently supports + :ref:`Self-hosted models `, :ref:`OCI GenAI models `, and :ref:`3rd party models `. See the :ref:`API reference ` for + more information. + +**How do I interact with data?** + + You can use the ``Datastore`` abstraction to interact with data structures. + See the :ref:`API reference ` for more information. + + +**Why should I implement my assistant as a config file rather than custom code?** + + To avoid custom steps as much as possible, as they are difficult to test and make it harder to understand the logic of the application. + Instead, you can use the base classes provided by WayFlow to create your assistant. diff --git a/26.1.2/_sources/core/howtoguides/agents.rst.txt b/26.1.2/_sources/core/howtoguides/agents.rst.txt new file mode 100644 index 000000000..e4a8fb9a8 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/agents.rst.txt @@ -0,0 +1,112 @@ +==================================== +How to Configure Agents Instructions +==================================== + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Agents <../tutorials/basic_agent>`. + +Agents can be configured to tackle many scenarios. +Proper configuration of their instructions is essential. + +In this how to guide, we will learn how to: + +- Configure the instructions of an :ref:`Agent `. +- Set up instructions that vary with each ``Conversation``. +- Maintain instructions that are consistently updated and refreshed. + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Basic implementation +==================== + +Assuming you need an agent to assist a user in writing articles, use the implementation below for that purpose: + +.. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-agent: + :end-before: .. end-agent + +Then execute it: + +.. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-execute: + :end-before: .. end-execute + +Sometimes, there is contextual information relevant to the conversation. +Assume a user is interacting with the assistant named "Jerry." +To make the assistant more context-aware, define a variable or expression in the ``custom_instruction`` Jinja template, and pass it when creating the conversation: + +.. note:: + + Jinja templating introduces security concerns that are addressed by WayFlow by restricting Jinja's rendering capabilities. + Please check our guide on :ref:`How to write secure prompts with Jinja templating ` for more information. + +.. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-conversation: + :end-before: .. end-conversation + +.. note:: + It is useful to use the same :ref:`Agent `, but change some part of the ``custom_instruction`` for each different conversation. + +Finally, incorporating dynamic context into the agent's instructions can significantly improve its responsiveness. +For example, the instructions can contain the current time making the agent more aware of the situation. +The time value is constantly changing, so you need to make sure it is always up-to-date. +To do this, use the ``ContextProvider``: + +.. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-context: + :end-before: .. end-context + +You successfully customized the prompt of your agent. + +Recap +===== + +In this guide, you learned how to configure :ref:`Agent ` instructions with: + +- pure text instructions; +- specific variables for each ``Conversation``; +- instructions with variables that needs to be always updated. + +.. collapse:: Below is the complete code from this guide. + + .. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-llm: + :end-before: .. end-llm + + .. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-agent: + :end-before: .. end-agent + + .. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-execute: + :end-before: .. end-execute + + .. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-conversation: + :end-before: .. end-conversation + + .. literalinclude:: ../code_examples/howto_agents.py + :language: python + :start-after: .. start-context: + :end-before: .. end-context + + +Next steps +========== + +Having learned how to configure agent instructions, you may now proceed to: + +- :doc:`How to Build Assistants with Tools ` +- :doc:`How to Use Agents in Flows ` diff --git a/26.1.2/_sources/core/howtoguides/catching_exceptions.rst.txt b/26.1.2/_sources/core/howtoguides/catching_exceptions.rst.txt new file mode 100644 index 000000000..9b970209e --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/catching_exceptions.rst.txt @@ -0,0 +1,167 @@ +.. _top-catchingexceptions: + +================================ +How to Catch Exceptions in Flows +================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_catchingexceptions.py + :link-alt: Catching exceptions how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +Exception handling is a crucial aspect of building robust and reliable software applications. +It allows a program to gracefully handle unexpected issues without crashing. +In WayFlow, exception handling can be achieved using the :ref:`CatchExceptionStep ` API. + +This guide shows you how to use this step to catch and process exceptions in a sub-flow: + +.. image:: ../_static/howto/catchexceptionstep.svg + :align: center + :scale: 100% + :alt: Simple Flow using a catch exception step + +.. seealso:: + The :ref:`RetryStep ` can be used to retry a sub-flow on specific criteria. See API documentation for more information. + + +Basic implementation +==================== + +To catch exceptions in a sub-flow, WayFlow offers the :ref:`CatchExceptionStep ` class. +This step is configured to run a flow and catches any exceptions that occur during its execution. + +The following example demonstrates the use of the ``CatchExceptionStep``. +Assuming you want to catch only ``ValueError`` exceptions. Specify them in the ``except_on`` parameter and define the branch name to which the flow will continue upon catching such exceptions. + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Define_Catch_Exception_Step + :end-before: .. end-##_Define_Catch_Exception_Step + +Once this is done, create the main flow and map each branch of the catch exception step to the next step. +In this example, the catch exception step has one branch for when a ``ValueError`` is caught (named ``VALUE_ERROR_BRANCH``), +and one default branch when no exception is raised (``Step.BRANCH_NEXT``, which is the only branch of the sub-flow). + +You can check the branch name of a step using the ``step.get_branches()`` function. + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Build_Exception_Handling_Flow + :end-before: .. end-##_Build_Exception_Handling_Flow + +Now you have a complete flow that takes different transitions depending on whether an exception was raised or not: + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Execute_Flow_With_Exceptions + :end-before: .. end-##_Execute_Flow_With_Exceptions + +.. tip:: + When developing flows, similarly to try-catch best practices in Python, we recommended to wrap only the steps that are likely to raise errors. + This approach helps to keep the flow organized and easier to maintain. + By wrapping only the error-prone steps, you can catch and handle specific exceptions more effectively, reducing the likelihood of masking other unexpected issues. + + +Common patterns +=============== + +Catching all exceptions +----------------------- + +To catch all exceptions and redirect to a shared branch, use the ``catch_all_exceptions`` parameter of the ``CatchExceptionStep`` class, and specify the transition of the branch ``CatchExceptionStep.DEFAULT_EXCEPTION_BRANCH`` as shown below: + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Catch_All_Exceptions + :end-before: .. end-##_Catch_All_Exceptions + + +Catching OCI model errors +------------------------- + +OCI models may raise a ``ServiceError`` when used (for example, when an inappropriate content is detected). +You can directly wrap the :ref:`PromptExecutionStep ` using the ``CatchExceptionStep`` and the helper method ``Flow.from_steps``, as follows: + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Handle_OCI_Service_Error + :end-before: .. end-##_Handle_OCI_Service_Error + + +Agent Spec Exporting/Loading +============================ + +You can export the flow configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_catchingexceptions.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_catchingexceptions.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_catchingexceptions.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginCatchExceptionNode`` + - ``PluginOutputMessageNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +In this guide, you learned how to do exception handling in WayFlow using the ``CatchExceptionStep`` class to: + +- catch specific exceptions with the ``except_on`` parameter; +- catch all exceptions with the ``catch_all_exceptions`` parameter. + +By following these steps and best practices, you can build more robust and reliable software applications using Flows. + +Having learned how to handle exceptions in flows, you may now proceed to :doc:`How to Create Conditional Transitions in Flows `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_catchingexceptions.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/conditional_flows.rst.txt b/26.1.2/_sources/core/howtoguides/conditional_flows.rst.txt new file mode 100644 index 000000000..e327c3ba7 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/conditional_flows.rst.txt @@ -0,0 +1,264 @@ +.. _top-howtoconditionaltransitions: + +============================================== +How to Create Conditional Transitions in Flows +============================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_branching.py + :link-alt: Conditional Transitions in Flows how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +Software applications utilize branching and conditionals to make decisions and respond dynamically +based on user inputs or data. This capability is essential for adapting to diverse scenarios, +ensuring a seamless and responsive user experience. + +WayFlow enables conditional transitions in Flows too. This guide demonstrates how to use +the :ref:`BranchingStep ` to execute different flows based on specific conditions. + +.. image:: ../_static/howto/branchingstep.svg + :align: center + :scale: 100% + :alt: Flow diagram of a simple branching step + +WayFlow offers additional APIs for managing conditional transitions, such as +:ref:`ChoiceSelectionStep `, :ref:`FlowExecutionStep `, +and :ref:`RetryStep `. For more information, refer to the API documentation. + + +Basic implementation +==================== + +Suppose there is a variable ``my_var`` that can be equal to ``"[SUCCESS]"`` or ``"[FAILURE]"``. +You want to perform different actions depending on its value. A ``BranchingStep`` can be used +to map each value to a corresponding branch: + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Branching_step + :end-before: .. end-##_Branching_step + +Once this is done, create the flow and map each branch to its corresponding next step. In this +example, the branching step has 2 branches based on the configuration (specified in the +``branch_name_mapping``), and also the default one (``BranchingStep.BRANCH_DEFAULT``). + +You can check the branch name of a step using the ``step.get_branches()`` function. + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Flow + :end-before: .. end-##_Flow + +.. note:: + Most steps only have a single next step, so you do not need to specify a transition dictionary, + and can just use a list with a single element. + + For steps with several branches (such as :ref:`BranchingStep `, :ref:`ChoiceSelectionStep `, + :ref:`RetryStep `, and :ref:`FlowExecutionStep `), you need to mapping each branch name to + the next step using an edge. Creating the flow will inform you if you are missing a branch in + the mapping. + + +You now have a flow which takes a different transition depending on the value of some variable: + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Execute + :end-before: .. end-##_Execute + +You now have the possibility to export your Flow as an Agent Spec configuration. The Agent Spec +configuration is a convenient serialized format that can be easily shared and stored. Additionally, +it allows execution in compatible environments. + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Export_to_Agent_Spec + :end-before: .. end-##_Export_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_branching.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_branching.yaml + :language: yaml + +You can now load back the configuration and execute it in the same manner as before exporting it. + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Load_and_execute_with_Agent_Spec + :end-before: .. end-##_Load_and_execute_with_Agent_Spec + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginRegexNode`` + - ``PluginOutputMessageNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Common patterns and best practices +================================== + + +Pattern 1: Branching if a token is present in a text +---------------------------------------------------- + +Most of the time, you will use :ref:`BranchingStep ` to branch out depending on whether a token is present +in a text (for example, whether an LLM generated a token ``[SUCCESS]`` or not). +To do this, pass :ref:`RegexExtractionStep ` before the :ref:`BranchingStep `: + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Branching_with_a_regular_expression + :end-before: .. end-##_Branching_with_a_regular_expression + + +You can apply this pattern for an LLM before producing a decision. +Generating a comprehensive textual review before providing a decision token can reduce hallucinations in LLM outputs. +This approach allows the model to contextualize its decision, leading to more accurate and reliable outcomes. + + +Pattern 2: Branching with more advanced expressions +--------------------------------------------------- + +For scenarios requiring branching based on more advanced conditions, consider using +:ref:`TemplateRenderingStep ` (which employs Jinja2) or +:ref:`ToolExecutionStep ` to evaluate conditions on variables. + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Branching_with_a_template + :end-before: .. end-##_Branching_with_a_template + +.. note:: + + Jinja templating introduces security concerns that are addressed by WayFlow by restricting Jinja's rendering capabilities. + Please check our guide on :ref:`How to write secure prompts with Jinja templating ` for more information. + +Pattern 3: Branching using an LLM +--------------------------------- + +To begin, configure an LLM. + +WayFlow supports several LLM API providers. Select an LLM from the options below to +proceed with the configuration. + +.. include:: ../_components/llm_config_tabs.rst + +You can implement branching logic determined by the LLM by using +:ref:`ChoiceSelectionStep `. To do so, pass the names and descriptions of the +potential next branches. + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Branching_with_an_LLM + :end-before: .. end-##_Branching_with_an_LLM + +.. tip:: + If needed, override the default template using the ``prompt_template`` argument. + +Pattern 4: Conditional branching with a sub-flow +------------------------------------------------ + +To implement branching based on multiple possible outcomes of a sub-flow, wrap it in +:ref:`FlowExecutionStep `. It will expose one branch per one possible end. +Mapping works the same as for :ref:`BranchingStep `: + +.. literalinclude:: ../code_examples/howto_branching.py + :language: python + :start-after: .. start-##_Branching_with_a_Subflow + :end-before: .. end-##_Branching_with_a_Subflow + + +Troubleshooting +=============== + +In case you forget to specify a branch for a step that has several sub-flows, the flow constructor +will inform you about the missing branch names: + +.. code-block:: python + + from wayflowcore.controlconnection import ControlFlowEdge + from wayflowcore.steps import BranchingStep, OutputMessageStep + from wayflowcore.flow import Flow + + branching_step = BranchingStep( + name="branching_step", + branch_name_mapping={ + "[SUCCESS]": "success", + "[FAILURE]": "failure", + }, + ) + success_step = OutputMessageStep("It was a success", name="success_step") + failure_step = OutputMessageStep("It was a failure", name="failure_step") + flow = Flow( + begin_step=branching_step, + control_flow_edges=[ + ControlFlowEdge( + source_step=branching_step, + destination_step=success_step, + source_branch="success", + ), + ControlFlowEdge( + source_step=branching_step, + destination_step=failure_step, + source_branch="failure", + ), + # Missing some control flow edges + ControlFlowEdge(source_step=success_step, destination_step=None), + ControlFlowEdge(source_step=failure_step, destination_step=None), + ], + ) + + # UserWarning: Missing edge for branch `default` of step ``. You only passed the following `control_flow_edges`: [ControlFlowEdge(source_step=, destination_step=, source_branch='success', __metadata_info__={}), ControlFlowEdge(source_step=, destination_step=, source_branch='failure', __metadata_info__={})]. The flow will raise at runtime if this branch is taken. + + + +Next steps +========== + +In this guide, you explored methods for implementing conditional branching within a Flow: + +- :ref:`BranchingStep `. +- :ref:`BranchingStep ` with pattern matching. +- more complex conditionals with :ref:`ToolExecutionStep `, or :ref:`TemplateRenderingStep `, and + :ref:`BranchingStep `. +- an LLM to decide on the condition using :ref:`ChoiceSelectionStep `. +- a sub-flow to handle the conditional logic using :ref:`FlowExecutionStep `. + +Having learned how to implement conditional branching in flows, you may now proceed to :doc:`Catching Exceptions ` to see how to ensure robustness in a ``Flow``. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the +full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_branching.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/create_a_tool_from_a_flow.rst.txt b/26.1.2/_sources/core/howtoguides/create_a_tool_from_a_flow.rst.txt new file mode 100644 index 000000000..7a55334a1 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/create_a_tool_from_a_flow.rst.txt @@ -0,0 +1,59 @@ +================================= +How to Create a Tool Using a Flow +================================= + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Tools <../api/tools>` + - :doc:`Building Assistants with Tools ` + +Equipping assistants with :doc:`Tools <../api/tools>` enhances their capabilities. +In WayFlow, tools can be defined in various ways. +One approach is to define a flow as the basis for the tool. +In this guide, you will see a basic example of how a flow is used to define a tool. + + +Defining the tool +================= + +In this guide, you will use an LLM. + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Now define a flow and pass additional information to describe the tool, including a name, description, and the output choice. + +.. literalinclude:: ../code_examples/howto_tool_from_flow.py + :language: python + :start-after: .. start: + :end-before: .. end-start + +.. note:: + The above example also works with more complex flows. However, only flows that do not yield are supported — meaning the flow must run to completion without pausing to request additional input from the user. + +.. tip:: + You can now use this tool like any other server tool, and pass it either to an :ref:`Agent ` or to a :ref:`ToolExecutionStep `. + + +Recap +===== + +In this guide, you learned how to create server tools from ``Flows`` by using the ``ServerTool.from_flow`` method. + +.. collapse:: Below is the complete code from this guide. + + .. literalinclude:: ../code_examples/howto_tool_from_flow.py + :language: python + :start-after: .. start: + :end-before: .. end-start + +Next steps +========== + +Having learned how to use tools in WayFlow, you may now proceed to :doc:`How to Build Assistants with Tools `. diff --git a/26.1.2/_sources/core/howtoguides/embeddingmodels_from_different_providers.rst.txt b/26.1.2/_sources/core/howtoguides/embeddingmodels_from_different_providers.rst.txt new file mode 100644 index 000000000..9583da105 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/embeddingmodels_from_different_providers.rst.txt @@ -0,0 +1,211 @@ +==================================================== +How to Use Embedding Models from Different Providers +==================================================== + +WayFlow supports several embedding model providers. The available embedding models are: + +- :ref:`OCIGenAIEmbeddingModel ` +- :ref:`OpenAIEmbeddingModel ` +- :ref:`VllmEmbeddingModel ` +- :ref:`OllamaEmbeddingModel ` + +Their configuration is specified directly to their respective class constructor. +This guide will show you how to configure embedding models from different providers with examples and notes on usage. + +Basic Implementation +-------------------- + +WayFlow provides multiple ways to work with embedding models. You can directly instantiate embedding model classes for maximum flexibility and control, which is the approach demonstrated in this guide. Each embedding model class extends the abstract ``EmbeddingModel`` base class and implements the required `embed` method. + +For the embedding models shown in this guide: + +- All models implement a consistent interface through the abstract ``EmbeddingModel`` base class +- Models generally require provider-specific configuration for authentication and endpoint access +- Security best practices are followed, with sensitive authentication data never being serialized +- The ``SerializableObject`` interface allows for saving and loading model configurations + +The following sections provide detailed information about each supported embedding model type, with examples showing how to instantiate and use them. + +.. _openaicompatible_embedding_custom_tls: + +Custom TLS Certificates for OpenAI-Compatible Embedding Endpoints +----------------------------------------------------------------- + +``OpenAICompatibleEmbeddingModel``, ``VllmEmbeddingModel`` and ``OllamaEmbeddingModel`` accept optional +TLS certificate parameters for HTTPS endpoints that use private certificate authorities or mutual TLS. + +.. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. openai-compatible-embedding-tls-start + :end-before: .. openai-compatible-embedding-tls-end + +Use ``ca_file`` to trust a custom certificate authority, and use ``key_file`` together with ``cert_file`` for mutual TLS (mTLS). +These certificate fields are runtime-only and are intentionally omitted from WayFlow serialization output. + +OCI GenAI Embedding Model +------------------------- + +`OCI GenAI Embedding Model `_ is powered by `OCI Generative AI `_. + +**Parameters** + +.. option:: model_id: str + + Name of the model to use. A list of the available models is given in + `Oracle OCI Documentation `_ + under the Model Retirement Dates (Embedding Models) section. + +.. option:: config: OCIClientConfig + + OCI client config to authenticate the OCI service. + See the below examples and :ref:`ociclientconfigclassesforauthentication` for the usage and more information. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. oci-embedding-start + :end-before: .. oci-embedding-end + +.. collapse:: Equivalent code example utilizing the OCIUserAuthenticationConfig class (API_KEY authentication without config/key files). + + WayFlow allows users to authenticate OCI GenAI service using a user API key without relying on a local config file and a key file. + + Instead of using a config file, the values of config parameters can be specified in the :ref:`OCIUserAuthenticationConfig `. + + .. literalinclude:: ../code_examples/example_oci_embeddings_userauthentication.py + :language: python + :start-after: .. start-embeddings-userauthenticationconfig: + :end-before: .. end-embeddings-userauthenticationconfig + + .. note:: + + The user authentication config parameters are sensitive information. This information will not be included when serializing a flow (there will be just an empty dictionary instead). + + You can create a client configuration with the user authentication configuration. + + .. literalinclude:: ../code_examples/example_oci_embeddings_userauthentication.py + :language: python + :start-after: .. start-embeddings-clientconfig: + :end-before: .. end-embeddings-clientconfig + + + Then create an ``OCIGenAIEmbeddingModel`` object: + + .. literalinclude:: ../code_examples/example_oci_embeddings_userauthentication.py + :language: python + :start-after: .. start-ocigenaiembeddingmodel: + :end-before: .. end-ocigenaiembeddingmodel + +**Notes** + +- Make sure to properly set up authentication configuration. +- Make sure that you have the ``oci>=2.134.0`` package installed. With your WayFlow environment activated, you can install the package as follows: + + .. code-block:: bash + + pip install oci>=2.134.0 + +.. important:: + If, when using the ``INSTANCE_PRINCIPAL``, the response of the model returns a ``404`` error, + check if your instance is listed in the dynamic group and has the right privileges. + Otherwise, ask someone with administrative privileges to grant your OCI Compute instance the ability to authenticate as an Instance Principal. + You need to have a Dynamic Group that includes the instance and a policy that allows this dynamic group to manage OCI GenAI services. + +OpenAI Embedding Model +---------------------- + +OpenAI Embedding Model is powered by `OpenAI `_. + +**Parameters** + +- **model_id** : str + Name of the model to use. Current supported models: ``text-embedding-3-small``, ``text-embedding-3-large``, and ``text-embedding-ada-002`` + (legacy model). + +- **api_key** : str, optional + The API key for authentication with OpenAI. If not provided, the value of the + ``OPENAI_API_KEY`` environment variable will be used, if set. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. openai-embedding-start + :end-before: .. openai-embedding-end + +vLLM Embedding Model +-------------------- + +`vLLM Embedding Model `_ is a model hosted with a vLLM server. + +**Parameters** + +- **url** : str + The complete URL of the vLLM server where the model is hosted (e.g., "http://localhost:8000" or "https://secure-vllm.example.com"). + Both HTTP and HTTPS protocols are supported. + +- **model_id** : str + Name of the model to use. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. vllm-embedding-start + :end-before: .. vllm-embedding-end + +**Notes** + +The vLLM embedding model makes HTTP/HTTPS POST requests to the /v1/embeddings endpoint of the specified URL, +following the OpenAI API format. When using HTTPS, certificate verification is performed by default. +For self-signed certificates, additional configuration may be needed at the application level. + +Ollama Embedding Model +---------------------- + +`Ollama Embedding Model `_ is powered by a locally hosted Ollama server. + +**Parameters** + +- **url** : str + The complete URL of the Ollama server where the model is hosted (e.g., "http://localhost:11434" or "https://ollama.example.com"). + Both HTTP and HTTPS protocols are supported. + +- **model_id** : str + Name of the model to use. A list of model names can be found `here `_. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. ollama-embedding-start + :end-before: .. ollama-embedding-end + +**Notes** + +The Ollama embedding model makes HTTP/HTTPS POST requests to the /v1/embeddings endpoint of the specified URL, +following the OpenAI API format. Ensure your Ollama server is properly configured with models that support embedding generation. +When using HTTPS, certificate verification is performed by default. For self-signed certificates, +additional configuration may be needed at the application level. + +Recap +----- + +This guide provides detailed descriptions of each embedding model type supported by WayFlow, demonstrating how to use both the configuration dictionary and direct instantiation methods for each model. +You can find below the complete code example presented in this guide: + +.. collapse:: Full Code + + .. literalinclude:: ../code_examples/example_initialize_embedding_models.py + :language: python + :start-after: .. recap: + :end-before: .. end-recap + + +Next steps +---------- + +Having learned how to configure and initialize embedding models from different providers, you may now proceed to: + +- :doc:`Datastores ` diff --git a/26.1.2/_sources/core/howtoguides/generation_config.rst.txt b/26.1.2/_sources/core/howtoguides/generation_config.rst.txt new file mode 100644 index 000000000..1925da05b --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/generation_config.rst.txt @@ -0,0 +1,229 @@ +.. _top-howto-generationconfig: + +=========================================================== +How to Specify the Generation Configuration when Using LLMs +=========================================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/example_generationconfig.py + :link-alt: Generation Configuration how-to script + + Python script/notebook for this guide. + +Generation parameters, such as temperature, top-p, the maximum number of output tokens, and per-token log-probabilities, are important for achieving the desired performance with Large Language Models (LLMs). +In WayFlow, these parameters can be configured with the :ref:`LlmGenerationConfig ` class. + +This guide will show you how to: + +- Configure the generation parameters for an agent. +- Configure the generation parameters for a flow. +- Request token log probabilities. +- Apply the generation configuration from a dictionary. +- Save a custom generation configuration. + +.. note:: + For a deeper understanding of the impact of each generation parameter, refer to the resources at the bottom of this page. + + +Basic implementation +==================== + +Configure the generation parameters for an agent +------------------------------------------------ + +Customizing the generation configuration for an agent requires the use of the following ``wayflowcore`` components. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Imports + :end-before: .. end-##_Imports + +The generation configuration can be specified when initializing the LLM using the :ref:`LlmGenerationConfig ` class. +This ensures that all the outputs generated by the agent will have the same generation configuration. + +The generation configuration dictionary can have the following arguments: + +- ``max_new_tokens``: controls the maximum numbers of tokens to generate, ignoring the number of tokens in the prompt; +- ``temperature``: controls the randomness of the output; +- ``top_p``: controls the randomness of the output; +- ``stop``: defines a list of stop words to indicate the LLM to stop generating; +- ``frequency_penalty``: controls the frequency of tokens generated. +- ``top_logprobs``: requests token-level log probabilities, including alternate candidates when the provider supports them. + +Additionally, the :ref:`LlmGenerationConfig ` offers the possibility to set a dictionary +of arbitrary parameters, called ``extra_args``, that will be sent as part of the llm generation call. +This allows specifying provider-specific parameters that might not be common to all. + +.. note:: + The extra parameters should never include sensitive information. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Define_the_llm_generation_configuration + :end-before: .. end-##_Define_the_llm_generation_configuration + +WayFlow supports several LLM API providers. +You can pass the ``generation_config`` for each of them. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +.. important:: + API keys should not be stored anywhere in the code. Use environment variables and/or tools such as `python-dotenv `_ + +Now, you can build an agent using the LLM as follows: + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Build_the_agent_and_run_it + :end-before: .. end-##_Build_the_agent_and_run_it + +Configure the generation parameters for a flow +---------------------------------------------- + +Customizing the generation configuration for a flow requires the use of the following ``wayflowcore`` components. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Import_what_is_needed_to_build_a_flow + :end-before: .. end-##_Import_what_is_needed_to_build_a_flow + +Refer to the previous section to learn how to configure the generation parameters when initializing an LLM +using the :ref:`LlmGenerationConfig ` class. + +You can then create a one-step flow using the :ref:`PromptExecutionStep ` step. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Build_the_flow_using_custom_generation_parameters + :end-before: .. end-##_Build_the_flow_using_custom_generation_parameters + +.. important:: + The ``generation_config`` parameter passed to the ``PromptExecutionStep`` overrides the LLM's original generation configuration. + +Advanced usage +============== + +The :ref:`LlmGenerationConfig ` class is a serializable object. It can be instantiated from a dictionary or saved to one, as you will see below. + + +.. _request_logprobs: + +Request token log probabilities +------------------------------- + +Use ``top_logprobs`` when you want the model to return token-level probabilities for generated text. +WayFlow stores those values on ``TextContent.logprobs`` for direct LLM calls, and the +:ref:`PromptExecutionStep ` also exposes them as an additional ``logprobs`` output. + +.. note:: + + ``top_logprobs`` is only available for raw text generation. + It is not supported with structured generation in :ref:`PromptExecutionStep `, + and support depends on the selected provider and model. + +For direct ``LlmModel`` calls, configure ``top_logprobs`` on the prompt and inspect the ``TextContent`` chunk: + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Request_logprobs_from_a_direct_llm_call + :end-before: .. end-##_Request_logprobs_from_a_direct_llm_call + +For flows, you can request logprobs directly on :ref:`PromptExecutionStep `. +When enabled, the step appends a ``logprobs`` output alongside the normal text output: + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Request_logprobs_from_a_flow_step + :end-before: .. end-##_Request_logprobs_from_a_flow_step + +Apply the generation configuration from a dictionary +---------------------------------------------------- + +If you have a generation configuration in a dictionary (for example, from a JSON or YAML file), +you can instantiate the :ref:`LlmGenerationConfig ` class as follows: + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Build_the_generation_configuration_from_dictionary + :end-before: .. end-##_Build_the_generation_configuration_from_dictionary + + +Save a custom generation configuration +-------------------------------------- + +If you would like to share your specific generation configuration, you can create a :ref:`LlmGenerationConfig ` +class instance and store it to a dictionary. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Export_a_generation_configuration_to_dictionary + :end-before: .. end-##_Export_a_generation_configuration_to_dictionary + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. +The following example exports the serialization of the flow defined above. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/example_generationconfig.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/example_generationconfig.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/example_generationconfig.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Next steps +========== + +Having learned how to specify the generation configuration, you may now proceed to: + +- :doc:`API Reference on LLM Models <../api/llmmodels>` +- :doc:`Build a Simple Conversational Assistant with Agents <../tutorials/basic_agent>` +- :doc:`How to Create Conditional Transitions in Flows ` +- :doc:`How to Build Assistants with Tools ` + +Some additional resources we recommend: + +- `HuggingFace - Generation with LLMs `_ +- `HuggingFace - Text generation strategies `_ + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/example_generationconfig.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_a2a_serving.rst.txt b/26.1.2/_sources/core/howtoguides/howto_a2a_serving.rst.txt new file mode 100644 index 000000000..45e95edc2 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_a2a_serving.rst.txt @@ -0,0 +1,141 @@ +.. _top-howtoa2aserving: + +========================================= +How to Serve Assistants with A2A Protocol +========================================= + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_a2a_serving.py + :link-alt: A2A Agent how-to script + + Python script/notebook for this guide. + +`A2A Protocol `_ is an open standard that defines how two agents can communicate +with each other. It covers both the serving and consumption aspects of agent interaction. + +This guide will show you how to serve a WayFlow assistant using this protocol either through the :ref:`A2AServer ` or from the command line. + +Basic implementation +==================== + +With the provided ``A2AServer``, you can: + +- Serve any conversational component in WayFlow, including :ref:`Agent `, :ref:`Flow `, :ref:`ManagerWorkers `, and :ref:`Swarm `. +- Serve from a serialized AgentSpec JSON/YAML string +- Serve from a path to an AgentSpec config file. + +In this guide, we start with serving a simple math agent equipped with a multiplication tool. + +To define the agent, you will need access to a large language model (LLM). +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Creating the agent +------------------ + +.. literalinclude:: ../code_examples/howto_a2a_serving.py + :language: python + :start-after: .. start-##_Create_the_agent + :end-before: .. end-##_Create_the_agent + +**API Reference:** :ref:`tool `, :ref:`Agent ` + +We recommend setting ``can_finish_conversation=True`` because, in A2A, each user request is treated as a `Task `_ that should complete once the request is processed. +Enabling this option allows the agent to return a *completed* status to clearly indicate that the task has finished. + +Serving the agent with ``A2AServer`` +------------------------------------ + +.. literalinclude:: ../code_examples/howto_a2a_serving.py + :language: python + :start-after: .. start-##_Serve_the_agent + :end-before: .. end-##_Serve_the_agent + +**API Reference:** :ref:`A2AServer ` + +You must specify the public URL where the agent will be reachable. +This URL is used to specify the agent's address in the Agent Card. + +When doing ``server.run``, the agent will be served at the specified ``host`` and ``port``. +The server exposes the following standard A2A endpoints: + +- ``/message/send``: for sending message requests +- ``/tasks/get``: for getting the information of a task +- ``/.well-known/agent-card.json``: for getting the agent card + +By default, when a client sends a message request, the server responds that the task has been submitted. +The client must then poll ``/tasks/get`` using the returned ``task_id``. + +If the client prefers to block and wait for the final response, it can set ``blocking=True`` when sending the message request. + +Serving the agent via CLI +------------------------- + +You can also serve an agent using its serialized AgentSpec configuration directly from the CLI: + +.. code-block:: bash + + wayflow serve \ + --api a2a \ + --agent-config agent.json \ + --tool-registry + +Since the agent uses a tool, you must pass the ``tool_registry``. +See the :ref:`API reference ` for a complete description of all arguments. + +Advanced usage +============== + +Storage configuration +--------------------- + +By default, ``InMemoryDatastore`` is used. +This is suitable for testing or local development, but not production. +For production, configure a persistent datastore through :ref:`ServerStorageConfig `. + +We support the following types of datastores: + +- ``InMemoryDatastore`` (not persistent) +- ``OracleDatastore`` (persistent) +- ``PostGresDatastore`` (persistent) + +Serving other WayFlow components +-------------------------------- + +We support serving all conversational components in WayFlow. + +For Flows: + +- Only Flows that *yield* are supported—that is, Flows containing :ref:`InputMessageStep ` or :ref:`AgentExecutionStep `. +- A Flow should include an :ref:`OutputMessageStep ` so that its final result can be returned to the client as a message. + +Below is example Flow that is valid for serving, along with how it can be served: + +.. literalinclude:: ../code_examples/howto_a2a_serving.py + :language: python + :start-after: .. start-##_Serve_a_flow + :end-before: .. end-##_Serve_a_flow + +**API Reference:** :ref:`Flow `, :ref:`InputMessageStep `, :ref:`AgentExecutionStep `, :ref:`OutputMessageStep ` + +Next steps +========== + +Now that you have learned how to serve WayFlow assistants using A2A protocol, you may proceed to :doc:`How to Build A2A Agents ` to learn how consume an A2A-served agent. + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_a2a_serving.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_a2aagent.rst.txt b/26.1.2/_sources/core/howtoguides/howto_a2aagent.rst.txt new file mode 100644 index 000000000..066fbf340 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_a2aagent.rst.txt @@ -0,0 +1,101 @@ +.. _top-howtoa2aagent: + +============================ +How to Connect to A2A Agents +============================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_a2aagent.py + :link-alt: A2A Agent how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLM configuration <../howtoguides/llm_from_different_providers>` + - :doc:`Using agents ` + + +`A2A Protocol `_ is an open standard that defines how two agents can communicate +with each other. It covers both the serving and consumption aspects of agent interaction. + +In this guide, you will learn how to connect to a remote agent using this protocol with the :ref:`A2AAgent ` +class from the ``wayflowcore`` package. + + +Basic usage +=========== + +To get started with an A2A agent, you need the URL of the remote server agent you wish to connect to. +Once you have this information, creating your A2A agent is straightforward and can be done in just a few lines of code: + +.. literalinclude:: ../code_examples/howto_a2aagent.py + :language: python + :start-after: .. start-##_Creating_the_agent + :end-before: .. end-##_Creating_the_agent + +Then, use the agent as shown below: + +.. literalinclude:: ../code_examples/howto_a2aagent.py + :language: python + :start-after: .. start-##_Running_the_agent + :end-before: .. end-##_Running_the_agent + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_a2aagent.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_a2aagent.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_a2aagent.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_a2aagent.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Next steps +========== + +Now that you have learned how to use A2A Agents in WayFlow, you may proceed to :doc:`How to Use Agents in Flows `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_a2aagent.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_agents_in_flows.rst.txt b/26.1.2/_sources/core/howtoguides/howto_agents_in_flows.rst.txt new file mode 100644 index 000000000..e9f6e031f --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_agents_in_flows.rst.txt @@ -0,0 +1,154 @@ +.. _top-howtoagentsinflows: + +========================== +How to Use Agents in Flows +========================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_agents_in_flows.py + :link-alt: Agents in Flows how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Agents <../tutorials/basic_agent>` + +Usually, flows serve as pipelines to ensure the robustness of agentic workloads. +Employing an agent for a specific task is desirable because of its ability to invoke tools when necessary and autonomously select parameters for certain actions. + +WayFlow enables the use of agents within flows, combining the predictability of flows with the adaptability of agents. +This guide demonstrates how to utilize the :ref:`AgentExecutionStep ` to embed an agent within a flow to execute a specific task. + +.. image:: ../_static/howto/agentstep.svg + :align: center + :scale: 120% + :alt: Flow diagram of a pipeline that uses agents + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Basic implementation +==================== + +Assuming you want to write an article. +Writing an article typically involves the following steps: + +1. Find the topic for the article. +2. Write the article. This stage typically includes looking for sources on the web, drafting the entire text, reviewing, checking grammar, sources, proofreading. +3. Submit the article by sending it via email to the editor. + +Steps 1 and 3 are straightforward and can be managed using standard procedures. +However, Step 2 involves complex tasks beyond a simple LLM generation, such as web browsing and content review. +To address this, you can use ``AgentExecutionStep`` that allows an agent to flexibly utilize tools for web browsing and article review. + +Assuming you already have the following tools to browse the web and to proofread the text: + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Define_the_tools + :end-before: .. end-##_Define_the_tools + +Continue creating the agent, specifying the agent's expected output using the ``outputs`` argument: + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Define_the_agent + :end-before: .. end-##_Define_the_agent + +The agent should operate within a flow without user interaction. +For that, set the ``caller_input_mode`` mode to ``CallerInputMode.NEVER``. + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Define_the_agent_step + :end-before: .. end-##_Define_the_agent_step + +Now finalize the entire flow: + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Define_the_Flow + :end-before: .. end-##_Define_the_Flow + +After completing the previous configurations, execute the flow. + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Execute_the_flow + :end-before: .. end-##_Execute_the_flow + +As expected, the final execution message should be the email to be sent to the editor. + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_agents_in_flows.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_agents_in_flows.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_agents_in_flows.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginInputMessageNode`` + - ``PluginOutputMessageNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Having learned how to use Agents inside Flows, you may now proceed to: + +- :doc:`How to Create Conditional Transitions in Flows ` to branch out depending on the agent's response. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_agents_in_flows.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_async.rst.txt b/26.1.2/_sources/core/howtoguides/howto_async.rst.txt new file mode 100644 index 000000000..6c12d26ad --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_async.rst.txt @@ -0,0 +1,200 @@ +.. _top-howtoasync: + +============================= +How to Use Asynchronous APIs +============================= + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_async.py + :link-alt: Asynchronous APIs how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLMs <../api/llmmodels>` + - :doc:`Agent <../api/agent>` + - :doc:`Flows <../api/flows>` + +Why async matters +================= + +Asynchronous (async) programming in Python lets you start operations that wait on I/O (network, disk, etc.) +without blocking the main thread, enabling high concurrency with a single event loop. +Async is ideal for I/O-bound workloads (LLM calls, HTTP requests, databases) and less useful for CPU-bound tasks, +which should run in worker threads or processes to avoid blocking the event loop. + +WayFlow provides asynchronous APIs across models (e.g., ``generate_async``), conversations (``execute_async``), +agents, and flows so you can compose concurrent, high-throughput pipelines using libraries such as ``anyio``. +Use async in the following cases: + +- Many parallel LLM requests +- Agents calling several tools that perform remote I/O +- Flows coordinating multiple steps concurrently + + +Basic implementation +==================== + +This section shows how to: + +1. Use an LLM asynchronously +2. Execute an agent and a flow asynchronously +3. Define tools properly for CPU-bound vs I/O-bound tasks +4. Run many agents concurrently with ``anyio`` +5. Understand when to still use synchronous APIs + + +For this tutorial, we will use a LLM. WayFlow supports several LLM API +providers, select a LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Using asynchronous APIs +----------------------- + +From synchronous code, wrap the coroutine with ``anyio.run(...)`` to execute it in an event loop without blocking. +Inside an ``async def`` function, you would instead write ``await ...``. +This pattern lets you fire off many I/O-bound calls concurrently and get much higher throughput than sync code. +For example, with an ``LlmModel``: + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Single_async_generation + :end-before: .. end-##_Single_async_generation + +Execute an Agent asynchronously +------------------------------- + +In async pipelines, you can now use ``execute_async`` to avoid head-of-line blocking. + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Async_Agent_execution + :end-before: .. end-##_Async_Agent_execution + +Execute a Flow asynchronously +----------------------------- + +Similarly, you can now run ``Flows`` asynchronously: + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Async_Flow_execution + :end-before: .. end-##_Async_Flow_execution + +Async tools vs sync tools +------------------------- + +:ref:`ServerTool ` 's callable can be synchronous or asynchronous. The ``tool`` decorator +can therefore be applied to both synchronous and asynchronous functionx. + +Use ``async`` tools for I/O-bound operations (HTTP calls, databases, storage) so they compose naturally +with the event loop. Keep CPU-bound work in synchronous functions, so that WayFlow automatically +runs them in worker threads in order to not block the event loop. + +.. tip:: + Avoid putting heavy CPU work inside an ``async def`` tool. If you must compute in an async context, + offload to a thread or keep it as a synchronous tool so WayFlow can schedule it efficiently. + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Define_tools_async_vs_sync + :end-before: .. end-##_Define_tools_async_vs_sync + +Use tools in an async Agent +--------------------------- + +Combine tools with an agent and run it asynchronously. + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Agent_with_async_tools + :end-before: .. end-##_Agent_with_async_tools + +Run many Agents concurrently +---------------------------- + +To scale throughput, use ``anyio.create_task_group()`` to start many agent runs concurrently. +Each task awaits its own ``execute_async`` call; the event loop interleaves I/O so all runs make progress. +You can bound concurrency by using semaphores if your backend has rate limits. + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Run_agents_concurrently + :end-before: .. end-##_Run_agents_concurrently + +Synchronous APIs in synchronous contexts +---------------------------------------- + +Synchronous APIs remain useful in simple scripts and batch jobs. Prefer them only when you are not inside +an event loop. If you call ``execute()`` or other sync APIs from async code, you risk blocking the loop; +WayFlow emits a warning and tells you which async method to use instead (for example, ``execute_async``). + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Synchronous_usage + :end-before: .. end-##_Synchronous_usage + +Agent Spec Exporting/Loading +============================ + +You can export the agent configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_async.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_async.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_async.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + + +Next steps +========== + +Having learned how to use the asynchronous APIs, you may now proceed to: + +- :doc:`Use Agents in Flows ` +- :doc:`Do Structured LLM Generation in Flows ` +- :doc:`Build Assistants with WayFlow Tools ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_async.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_build_assistants_with_tools.rst.txt b/26.1.2/_sources/core/howtoguides/howto_build_assistants_with_tools.rst.txt new file mode 100644 index 000000000..73b0ceb5a --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_build_assistants_with_tools.rst.txt @@ -0,0 +1,336 @@ +.. _tool_use: + +================================== +How to Build Assistants with Tools +================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_tooluse.py + :link-alt: Tool use how-to script + + Python script/notebook for this guide. + + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Agents <../tutorials/basic_agent>` + +Equipping assistants with :doc:`Tools <../api/tools>` enhances their capabilities. +In WayFlow, tools can be used in both conversational assistants (also known as Agents) as well as in Flows by using the ``ToolExecutionStep`` class. + +WayFlow supports **server-side**, **client-side** tools as well as **remote-tools**. + +.. seealso:: + + This guide focuses on using server/client tools. Read the :ref:`API Documentation ` to learn + more about the ``RemoteTool``. + +.. image:: ../_static/howto/types_of_tools.svg + :align: center + :scale: 80% + + +In this guide, you will build a **PDF summarizer** using the different types of supported tools, both within a flow and with an agent. + + +Imports and LLM configuration +============================= + +To get started, first import the ``PyPDF`` library. +First, install the PyPDF library: + +.. code:: bash + + $ pip install pypdf + +Building LLM-powered assistants with tools in WayFlow requires the following imports. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Imports_for_this_guide + :end-before: .. end-##_Imports_for_this_guide + +In this guide, you will use an LLM. + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +.. note:: + API keys should not be stored anywhere in the code. Use environment variables or tools such as `python-dotenv `_. + + +Helper functions +================ + +The underlying tool used in this example is a PDF parser tool. The tool: + +- loads a PDF file; +- reads the content of the pages; +- returns the extracted text. + +The ``PyPDFLoader`` API from the `langchain_community Python library `_ is used for this purpose. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Defining_some_helper_functions + :end-before: .. end-##_Defining_some_helper_functions + + +.. collapse:: Click to see the PDF content used in this example. + + .. literalinclude:: ../_static/howto/example_document.md + :language: markdown + +\ + +.. _overview-of-tool-types: + +Overview of the types of tools +============================== + +This section covers how to use the following tools: + +* ``@tool`` decorator - The simplest way to create server-side tools by decorating Python functions. +* ``ServerTool`` - For tools to be executed on the server side. Use this for tools running within the WayFlow environment, including local execution. +* ``ClientTool`` - For tools to be executed on the client application. + + +Using the @tool Decorator +------------------------- + +WayFlow provides a convenient ``@tool`` decorator that simplifies the creation of server-side tools. By decorating a Python function with ``@tool``, you automatically convert it into a ``ServerTool`` object ready to be used in your Flows and Agents. + +The decorator automatically extracts information from the function: + +- The function name becomes the tool name +- The function docstring becomes the tool description +- Type annotations and parameter docstrings define input parameters +- Return type annotations define the output type + +In the example below, we show a few options for how to create a server-side tool using the ``@tool`` decorator: + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Defining_a_tool_using_the_tool_decorator + :end-before: .. end-##_Defining_a_tool_using_the_tool_decorator + +In the above example, the decorated function ``read_pdf_server_tool`` is transformed into a ``ServerTool`` object. +The tool's name is derived from the function name, the description from the docstring, and the input parameters and output type from the type annotations. + +.. tip:: + + You can set the ``description_mode`` parameter of the ``@tool`` decorator to ``only_docstring`` to use the parameter signature information from + the docstrings instead of having to manually define them using ``Annotated[Type, "description"]``. + + +Creating a ServerTool +--------------------- + +The ``ServerTool`` is defined by specifying: + +- A tool name +- A tool description +- Input parameters, including names, types, and optional default values. +- A Python callable, the function executed by the tool. +- The output type. + +In the example below, the tool takes two input parameters, one of which is optional, and returns a string. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Defining_a_tool_using_the_ServerTool + :end-before: .. end-##_Defining_a_tool_using_the_ServerTool + + +Creating a ClientTool +--------------------- + +The ``ClientTool`` is defined similarly to a ``ServerTool``, except that it does not include a Python callable in its definition. +When executed, a ``ClientTool`` returns a ``ToolRequest``, which must be executed on the client side. +The client then sends the execution result back to the assistant. + +In the following example, the tool execution function is also defined. +This function should be implemented based on the specific requirements of the assistant developer. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Defining_a_tool_using_the_ClientTool + :end-before: # .. end-##_Defining_a_tool_using_the_ClientTool + + +Building Flows with Tools using the `ToolExecutionStep` +======================================================= + +Executing tools in Flows can be done using the :ref:`ToolExecutionStep `. +The step simply requires the user to specify the tool to execute when the step is invoked. + +Once the tool execution step is defined, the Flow can be constructed as usual. +For more information, refer to the tutorial on :doc:`Flows <../tutorials/basic_flow>`. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Defining_a_build_flow_helper_function + :end-before: # .. end-##_Defining_a_build_flow_helper_function + + +Executing ServerTool with a Flow +-------------------------------- + +When using the ``ServerTool``, the tool execution is performed on the server side. +As a consequence, the flow can be executed end-to-end with a single ``execute`` instruction. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Creating_and_running_a_flow_with_a_server_tool + :end-before: # .. end-##_Creating_and_running_a_flow_with_a_server_tool + + +.. collapse:: Click to see the summarized PDF content. + + Here is the summarized PDF: + + Oracle Corporation is an American multinational computer technology company headquartered + in Austin, Texas. It sells database software, cloud computing software and hardware, and + enterprise software products including ERP, HCM, CRM, and SCM software. + +\ + +Executing ClientTool with a Flow +-------------------------------- + +When using a ``ClientTool``, the tool execution is performed on the client side. +Upon request, the assistant sends a ``ToolRequest`` to the client, which is responsible for executing the tool. +Once completed, the client returns a ``ToolResult`` to the assistant, allowing it to continue execution until the task is complete. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Creating_and_running_a_flow_with_a_client_tool + :end-before: # .. end-##_Creating_and_running_a_flow_with_a_client_tool + + +Building Agents with Tools +========================== + +Agents can be equipped with tools by specifying the list of tools the agent can access. + +You do not need to mention the tools in the agent’s ``custom_instruction``, as tool descriptions are automatically added internally. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Defining_a_build_agent_helper_function + :end-before: # .. end-##_Defining_a_build_agent_helper_function + + +.. note:: + :ref:`Agents ` can also be equipped with other flows and agents. This topic will be covered in a dedicated tutorial. + + +Executing ServerTool with an Agent +---------------------------------- + +Similar to executing a Flow with a ``ServerTool``, Agents can be executed end-to-end using a single ``execute`` instruction. +The key difference is that the file path is provided as a conversation message rather than as a flow input. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Creating_and_running_an_agent_with_a_server_tool + :end-before: # .. end-##_Creating_and_running_an_agent_with_a_server_tool + + +.. important:: + In this case, the LLM must correctly generate the tool call with the file path as an input parameter. Smaller LLMs may struggle to reproduce the path accurately. + In general, assistant developers should try to avoid having LLMs to manipulate complex strings. + + +Executing ClientTool with an Agent +---------------------------------- + +Similar to executing a Flow with a ``ClientTool``, the tool request must be handled on the client side. +The only difference is that the file path is provided as a conversation message instead of as a flow input. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: # .. start-##_Creating_and_running_an_agent_with_a_client_tool + :end-before: # .. end-##_Creating_and_running_an_agent_with_a_client_tool + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_tooluse.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_tooluse.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_tooluse.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginPromptTemplate`` + - ``PluginRemoveEmptyNonUserMessageTransform`` + - ``ExtendedAgent`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + + +Next steps +========== + +Having learned how to use ``ServerTool`` and ``ClientTool`` with assistants in WayFlow, you may now proceed to: + +- :doc:`How to Create Tools with Multiple Outputs ` +- :doc:`How to Install and Use Ollama ` +- :doc:`How to Connect Assistants to Data `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_tooluse.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_conversation_evaluation.rst.txt b/26.1.2/_sources/core/howtoguides/howto_conversation_evaluation.rst.txt new file mode 100644 index 000000000..c7b16e809 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_conversation_evaluation.rst.txt @@ -0,0 +1,131 @@ +.. _top-howtoconversationevaluation: + +======================================= +How to Evaluate Assistant Conversations +======================================= + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_conversation_evaluation.py + :link-alt: Conversation Evaluation how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Agents <../tutorials/basic_agent>` + +Evaluating the robustness and performance of assistants requires careful conversation assessment. +The :ref:`ConversationEvaluator ` API in WayFlow enables evaluation of conversations +using LLM-powered criteria—helping you find weaknesses and improve your assistants. + +This guide demonstrates the process of constructing, scoring, and evaluating a conversation. + +.. image:: ../_static/howto/conversation_evaluator.png + :align: center + :scale: 40% + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Basic implementation +==================== + +Assume you want to evaluate the following assistant conversation, which purposefully exhibits poor assistant performance. + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Define_the_conversation + :end-before: .. end-##_Define_the_conversation + +The conversation alternates user and assistant messages, simulating a scenario with misunderstandings and wrong information. + +In a production context, you system would be collecting conversations, and you would evaluate +then offline. You can use serialization to serialize conversations easily in your production +environment, and reload them later for offline evaluation: + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Serialize_and_Deserialize_the_conversation + :end-before: .. end-##_Serialize_and_Deserialize_the_conversation + + +Defining the LLM to use as a judge +================================== + +We will need a LLM to judge the conversations. The first step is to instantiate an LLM supported by WayFlow. + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Define_the_llm + :end-before: .. end-##_Define_the_llm + +Defining scoring criteria +========================= + +The :ref:`ConversationScorer ` is the component responsible for scoring the conversation according to specific criteria. +Currently, two scorers are supported in ``wayflowcore``: + +- The :ref:`UsefulnessScorer ` score estimates the overall usefulness of the assistant from the conversation. It uses criteria such as: + - The task completion efficiency: does it seem like the assistant is able to complete the tasks? + - The level of proactiveness: is the assistant able to anticipate the user needs? + - The ambiguity detection capability: does the assistant often requires clarification or is more autonomous? +- The :ref:`UserHappinessScorer ` score estimates the level of happiness / frustration of the user from the conversation. It uses criteria such as: + - The query repetition frequency: does the user need to repeat their questions? + - The misinterpretation of user intent: is there misinterpretation from the assistant? + - The conversation flow disruption: does the conversation flow seamlessly or is severely disrupted? + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Define_the_scorers + :end-before: .. end-##_Define_the_scorers + +You can, or course, implement your own versions for your specific use-case, by respecting the +:ref:`ConversationScorer ` APIs. + +Setting up the evaluator +======================== + +The :ref:`ConversationEvaluator ` combines scorers and applies them to the provided conversation(s): + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Define_the_conversation_evaluator + :end-before: .. end-##_Define_the_conversation_evaluator + +Running the evaluation +====================== + +Trigger the evaluation and inspect the scoring DataFrame as output: + +.. literalinclude:: ../code_examples/howto_conversation_evaluation.py + :language: python + :start-after: .. start-##_Execute_the_evaluation + :end-before: .. end-##_Execute_the_evaluation + +The result is a table where each scorer provides a score for each conversation. + + +Next steps +========== + +After learning to use ``ConversationEvaluator`` to assess conversations, proceed to :doc:`Perform Assistant Evaluation ` for more advanced evaluation techniques. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide, or view it below. + +.. literalinclude:: ../end_to_end_code_examples/howto_conversation_evaluation.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_data_synthesis.rst.txt b/26.1.2/_sources/core/howtoguides/howto_data_synthesis.rst.txt new file mode 100644 index 000000000..e4ee7e2d2 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_data_synthesis.rst.txt @@ -0,0 +1,280 @@ +======================================== +How to Perform Data Synthesis in WayFlow +======================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_data_synthesis.py + :link-alt: Data Synthesis how-to script + + Python script/notebook for this guide. + +This guide provides practical and adaptable methods for generating synthetic datasets tailored to a range of real-world use cases. Use it when you need to create evaluation datasets for model testing, sample data for demonstrations, or address data privacy requirements. You will find reproducible workflows and techniques that you can flexibly adapt to your own data schema and constraints. + +Each synthesis approach is presented in relation to specific feature requirements. Most examples start from a seed dataset to capture authentic data characteristics. However, you can also apply these methods without real data by specifying a target schema and using tools such as *Faker*. With this guide, you can generate synthetic data that balances statistical realism with project flexibility. + +Prerequisites and setup +======================= + +- **Desired output schema:** Define the fields and features your synthetic dataset should contain. Specify each field's type and any necessary constraints. + +- **Seed dataset (optional, recommended):** + + - Use a seed dataset if you want to preserve real-world distributional properties, such as means, variances, or category frequencies. + + - A seed dataset is also essential if you need to interpolate or generate synthetic values conditioned on, or resembling, existing feature values (for example, generating synthetic ages based on a real distribution). + +- **No seed dataset?** + + If you do not have real data to start with, you can use the *Faker* package (`Faker `__). *Faker* provides many generators for structured data such as names, addresses, dates, countries, and phone numbers. Use this to quickly create dummy datasets with any schema, though the values will not reflect empirical sources. + +.. note:: + Choosing whether to use a seed dataset or a synthetic generator depends on your use case. + + - For data realism and statistical fidelity, start from a seed dataset. + - For flexible or arbitrary data structures without statistical constraints, use *Faker* or similar tools. + +Target schema and feature properties +==================================== + +The following table summarizes the type, synthesis method, and key properties or constraints for each feature: + ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| Feature | Type | Interp./Extrap. | Method or constraint | Distribution target | ++=======================+========+=====================+===========================================+======================+ +| customer_type | cat | Interpolation | Sampled with empirical (seed) univariate | Preserve univariate | +| | | | distribution | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| customer_name | string | Extrapolation | Use ``fake.name()`` if individual, else | / | +| | | | ``fake.company()`` | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| region | cat | Interpolation | Choice among [EMEA, JAPAC, LAD, NA] | / | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| country | cat | Interpolation | Based on mapping from region; randomly | Coherency: region | +| | | | select among possible countries in region | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| customer_net_worth | num | Interpolation | Sampled from joint empirical distribution | Joint: region | +| | | | with region | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| loan_value | num | Interpolation | Sampled from joint empirical distribution | Joint: region | +| | | | with region | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| loan_proposed_interest| num | Interpolation | Sampled from joint empirical distribution | Joint: region | +| | | | with region | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| loan_reason | cat | Interpolation | Sampled with empirical (seed) univariate | Preserve univariate | +| | | | distribution | | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ +| justification | string | Extrapolation | Long text generation | Coherency with other | +| | | | | fields | ++-----------------------+--------+---------------------+-------------------------------------------+----------------------+ + +- **Interp./Extrap.:** Indicates if the feature is synthesized from existing (seed) values (interpolation) or from new, plausible values (extrapolation). +- **Method or constraint:** Specifies any specialized synthesis function or logic, including use of *Faker*. + +Load and inspect the seed data +============================== + +Begin by loading your seed dataset. This data provides distributions, feature values, and coherency patterns for synthetic data generation. For demonstration, the following example defines a small seed dataset matching the target schema above, but you can use your own data. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-data: + :end-before: .. end-data + +Data synthesis +============== + +Synthesis functions +^^^^^^^^^^^^^^^^^^^ + +Define Python functions to generate feature values according to empirical distributions and project-specific constraints. + +1. ``fake_categorical``: Sample a categorical column by seed frequency + It uses NumPy to reproduce marginal distributions. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-fake_categorical: + :end-before: .. end-fake_categorical + +2. ``fake_joint_numerical``: Sample a numerical feature by joint distribution + It samples values for a numerical feature conditional on one or more categorical features (for example, ``customer_net_worth`` conditioned on region). + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-fake_joint_numerical: + :end-before: .. end-fake_joint_numerical + +Structured data generation +========================== + +Each section below demonstrates how to synthesize features while upholding a specific property. After each example, synthesize all other features by specification. + +Reproducibility +^^^^^^^^^^^^^^^ + +Set seeds for all relevant random number generators to make synthesis repeatable. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-seeds: + :end-before: .. end-seeds + +Generation configuration +^^^^^^^^^^^^^^^^^^^^^^^^ + +Configure the synthetic data generation process. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-generation_configuration: + :end-before: .. end-generation_configuration + +Univariate distribution preservation example (region) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Sample the ``region`` column according to the frequency distribution in the seed dataset. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-univariate_distribution: + :end-before: .. end-univariate_distribution + +Coherency constraint example (region vs. country) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +``country`` and ``region`` must be coherent: each country should map to its correct business region. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-coherency_constraint: + :end-before: .. end-coherency_constraint + +Joint distribution example (customer_net_worth | region) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Synthesize ``customer_net_worth`` values respecting the empirical joint distribution over ``region``. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-joint_distribution: + :end-before: .. end-joint_distribution + +Extrapolation (new values) example (customer_name) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use *Faker* to synthesize new individual names or company names, even if not present in the seed. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-extrapolation: + :end-before: .. end-extrapolation + +Synthesizing remaining structured features according to property table +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +With each property demonstrated, now synthesize the remaining structured features as specified above. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-full_synthesis: + :end-before: .. end-full_synthesis + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-full_synthesis_flow: + :end-before: .. end-full_synthesis_flow + +Verification: sanity checks and distribution comparison +======================================================= + +Compare value counts and distributions for select features, and compare them to the original seed dataset. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-verification: + :end-before: .. end-verification + +Long text generation +==================== +To generate realistic business justifications for each record, you will use a language model (LLM) as part of your synthesis workflow. This section explains the configuration of the LLM, describes the prompts and parsing logic, outlines the flow definition, and demonstrates parallelized generation and validation. + +LLM selection and initialization +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +For long text generation, you need to use an LLM. +WayFlow supports several LLM providers. Select and configure your LLM below: + +.. include:: ../_components/llm_config_tabs.rst + +Prompt and I/O constants +^^^^^^^^^^^^^^^^^^^^^^^^ +Begin by defining the necessary output constants and prompts used to instruct the LLM and structure the flow: + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-long_text_generation: + :end-before: .. end-long_text_generation + +Parsing functions +^^^^^^^^^^^^^^^^^ +Parsing functions are responsible for extracting and normalizing the LLM outputs produced during justification and validation steps: + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-parsing_functions: + :end-before: .. end-parsing_functions + +Justification generation flow definition +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +This flow defines a multi-step pipeline for generation, parsing, validation, and conditional retry of business justifications: + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-justification_generation_flow: + :end-before: .. end-justification_generation_flow + +Batch justification generation and validation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Finally, apply the justification generation flow in parallel over your dataset using a `MapStep` (for more information see :doc:`How to Do Map and Reduce Operations in Flows `). This enables efficient batch generation and validation: + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-justification_generation: + :end-before: .. end-justification_generation + +Synthetic dataset export +======================== + +Save the synthesized dataset for downstream use. + +.. literalinclude:: ../code_examples/howto_data_synthesis.py + :language: python + :start-after: .. start-exporting_synthetic_dataset: + :end-before: .. end-exporting_synthetic_dataset + +Recap +===== + +This guide demonstrated property-respecting structured data synthesis, including logic for univariate, joint, interpolated, extrapolated, and coherent features. It also described long text generation and validation with a language model as a judge, including conditional retries. By following these workflows and recommendations, you can create realistic, flexible synthetic datasets to support your WayFlow projects. + + +Next steps +========== + +Having learned how to synthesize data in WayFlow, you may now proceed to :doc:`How to Connect Assistants to Your Data ` to learn how to integrate your synthesized datasets with your assistants for downstream applications and enhanced testing. + + +Full code +========= + +You can copy the full code for this guide below. + +.. literalinclude:: ../end_to_end_code_examples/howto_data_synthesis.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_datastores.rst.txt b/26.1.2/_sources/core/howtoguides/howto_datastores.rst.txt new file mode 100644 index 000000000..5425a2e29 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_datastores.rst.txt @@ -0,0 +1,493 @@ +.. _top-howtodatastores: + +====================================== +How to Connect Assistants to Your Data +====================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download In-Memory Script + :link: ../end_to_end_code_examples/howto_datastores.py + :link-alt: In-memory datastore how-to script + + Python script/notebook for the In-Memory Datastore example in this guide. + .. grid-item-card:: |python-icon| Download Oracle Database Script + :link: ../end_to_end_code_examples/howto_connect_to_oracle_database.py + :link-alt: Oracle Database datastore how-to script + + Python script/notebook for Oracle Database Datastore example in this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with + + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Flows <../tutorials/basic_flow>` + + +Agents rely on access to relevant data to function effectively. +Without a steady stream of high-quality data, AI models are unable to learn, reason, or make informed decisions. +Connecting an Agent or Flow to data sources is therefore a critical step in developing functional AI systems. +This connection enables agents to perceive their environment, update their knowledge or the underlying data source, and adapt to changing conditions. + +In this tutorial, you will: + +- **Define Entities** that an Agent or Flow can access and manipulate. +- **Populate a Datastore** with entities for development and testing. +- **Use Datastores in Flows and Agents** to create two types of inventory management assistants. + +To ensure reproducibility of this tutorial, you will use an in-memory data source. +Check out the section :ref:`Using Oracle Database Datastore ` to see how to configure an +Oracle Database connection for persistent storage. + +Concepts shown in this guide +============================ + +- :ref:`Entities ` to model data +- :ref:`Datastore ` and :ref:`InMemoryDatastore ` to manipulate collections of data +- Steps to use datastores in Agents and Flows (:ref:`DatastoreListStep `, :ref:`DatastoreCreateStep `, :ref:`DatastoreUpdateStep `, :ref:`DatastoreDeleteStep `) + +.. note:: + The :ref:`InMemoryDatastore ` is mainly suitable for testing and development, + or other use-cases where data persistence across assistants and conversations is not a requirement. + For production use-cases, the :ref:`OracleDatabaseDatastore ` provides a + robust and scalable persistence layer in Oracle Database. + + Note that there are a few key differences between an in-memory and a database ``Datastore``: + + - With database Datastores, all tables relevant to the assistant must already be created in the database prior to connecting to it. + - You may choose to only model a subset of the tables available in the database via the :ref:`Entity ` construct. + - Database Datastores offer an additional ``query`` method (and the corresponding :ref:`DatastoreQueryStep `), + that enables flexible execution of SQL queries that cannot be modelled by the ``list`` operation on the in-memory datastore + +Datastores in Flows +=================== + +In this section, you will build a simple Flow that performs operations on an inventory database based on user input. +This Flow helps users keep product descriptions up to date by leveraging an LLM for the creative writing component. + +Step 1. Add imports and LLM configuration +----------------------------------------- + +Import the required packages: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Imports + :end-before: .. end-##_Imports + +In this assistant, you need to use an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +.. important:: + API keys should not be stored anywhere in the code. Use environment variables and/or tools such as `python-dotenv `_. + +Step 2. Define data +------------------- + +Start by defining the schema for your data using the :ref:`Entity ` construct. +In this example, you will manage products in an inventory, so a single collection is sufficient. +Datastores also support managing multiple collections at the same time if needed. + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_entities + :end-before: .. end-##_Create_entities + +Next, create an :ref:`InMemoryDatastore `. +For simplicity, you will use dummy data: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_datastore + :end-before: .. end-##_Create_datastore + +Step 3. Create datastore steps +------------------------------ + +Now that the Datastore is set up, create steps to perform different operations in the flow. +In this case, the assistant only needs to retrieve the current description of products and update it. +See :ref:`the list of all available Datastore steps ` for more details. + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_Datastore_step + :end-before: .. end-##_Create_Datastore_step + +Key points: + +- Use ``where`` to filter which product to list and update. Configure it with a variable so that the user can dynamically choose the product title. +- In the ``DatastoreListStep``, ``limit`` and ``unpack_single_entity_from_list`` are used to assume that product titles are unique, making the output concise and easy to handle. + +The input to the ``DatastoreListStep`` is the title of the product to retrieve, and the output is a single object containing the corresponding product data. +The input to the ``DatastoreUpdateStep`` includes both the title of the product to update and the updates to apply, in the form of a dictionary containing the properties and the corresponding values. +The output will be the new properties that were updated. + +Step 4. Create the Flow +----------------------- + +Now define the :ref:`Flow ` for this assistant. + +After the user enters which product they want to update, and how the description should be updated, the datastore is queried to find the matching product. +The LLM is prompted with the product details and the user's instructions. +The output is used to update the data, and the new result is returned back to the user. + +.. collapse:: Click to see the rest of the code + + .. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_flow + :end-before: .. end-##_Create_flow + +Note the use of structured generation in the ``PromptExecutionStep`` (via the ``output_descriptors`` parameter). +This ensures that the LLM generates exactly the structure expected by the ``DatastoreUpdateStep``. + +Finally, verify that the Flow works: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Execute_flow + :end-before: .. end-##_Execute_flow + + +Datastores in Agents +==================== + +This section assumes you have completed the previous steps on using datastores in Flows. + +The Flow you built earlier is quite helpful and reliable because it performs a single, specialized task. +Next, you will see how to define an Agent for inventory management when the task is not defined in advance. +This Agent will be able to interpret the user's requests and autonomously decide which actions on the ``Datastore`` are required to fulfill each task. + +Step 1. Add imports and LLM configuration +----------------------------------------- + +Add the additional imports needed to use Agents: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Import_agents + :end-before: .. end-##_Import_agents + +Step 2. Create Datastore Flows for an Agent +------------------------------------------- + +To use the ``Datastore`` in an agent, create flows for the different operations you want the agent to perform. +In the simplest setup, you can define one flow per basic ``Datastore`` operation. +The agent will then determine the correct sequence of actions to achieve the user's goal. + +Define flows for: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_agent_flows + :end-before: .. end-##_Create_agent_flows + +Notice the provided descriptions for the flows to help the agent understand the objective of each operation. + +Additionally, you can incorporate more complex behaviors into these flows. +For example, you could ask for user confirmation before deleting entities, or you could provide the user with an overview, and an editing option of the updates made by the agent before they are applied. + +Step 3. Create the Agent +------------------------ + +Finally, create the inventory management agent by combining the LLM, the datastore flows, and a custom instruction: + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :linenos: + :start-after: .. start-##_Create_the_agent + :end-before: .. end-##_Create_the_agent + +This agent can now respond to the user and perform actions on the data on their behalf. + +Refer to the :doc:`WayFlow Agents Tutorial <../tutorials/basic_agent>` to see how to run this Agent. + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_datastores.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_datastores.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_datastores.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginPromptTemplate`` + - ``PluginDatastoreCreateNode`` + - ``PluginDatastoreUpdateNode`` + - ``PluginDatastoreListNode`` + - ``PluginDatastoreDeleteNode`` + - ``PluginJsonToolOutputParser`` + - ``PluginRemoveEmptyNonUserMessageTransform`` + - ``PluginCoalesceSystemMessagesTransform`` + - ``PluginLlamaMergeToolRequestAndCallsTransform`` + - ``PluginInMemoryDatastore`` + - ``ExtendedAgent`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +.. _using-oracle-datastore: + +Using Oracle Database Datastore +=============================== + +This guide mirrors the earlier **in-memory** demo, but leverages Oracle Database +(Autonomous Database on OCI or an on-prem instance) for persistent storage. + +.. admonition:: Prerequisites + :class: tip + + * An Oracle Database instance, reachable from your development machine + * Wallet files and/or database credentials for this instance + * A running LLM endpoint (the examples uses vLLM, but any provider works) + +Step 1. Configure the connection to the Database +------------------------------------------------ + +WayFlow supports two secure transport mechanisms for connecting to Oracle databases. + +When using **TLS** (one-way TLS) the database presents its certificate, the client verifies it, +and the user authenticates with username and password. + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: + :start-after: .. start-##_TLS_Connection + :end-before: .. end-##_TLS_Connection + +============== ============================================================= +**Parameter** **Meaning** +-------------- ------------------------------------------------------------- +``user`` Database username (e.g., ``ADMIN``). +``password`` Password for ``user``. +``dsn`` Easy-connect string or TNS alias identifying the service. + (e.g., ``adb.us-ashburn-1.oraclecloud.com:1522/xyz_high``) +============== ============================================================= + +When using **mTLS** (mutual TLS) both sides exchange certificates: the client proves its identity +with a wallet (client cert + private key) in addition to the username and password. +This gives stronger, certificate-based client authentication and is often required for +Oracle Autonomous Database in "Require mTLS" mode. + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: + :start-after: .. start-##_mTLS_Connection + :end-before: .. end-##_mTLS_Connection + +====================== ============================================================= +**Parameter** **Meaning** +---------------------- ------------------------------------------------------------- +``user`` Database username (e.g., ``ADMIN``). +``password`` Password for ``user``. +``dsn`` Easy-connect string or TNS alias identifying the service. + Example: + ``adb.us-ashburn-1.oraclecloud.com:1522/xyz_high`` +``config_dir`` Directory containing ``sqlnet.ora`` / ``tnsnames.ora``. + when you want to reference a + TNS alias (e.g. ``_high``) instead of a raw DSN. +``wallet_location`` Path to the Oracle Wallet directory that contains + ``cwallet.sso`` / ``ewallet.p12``. +``wallet_password`` Password that protects the wallet’s private key. +====================== ============================================================= + +.. important:: + Do **not** hard-code any database credentials or sensitive connection details directly in your code. + Please refer to our :doc:`Security Guidelines <../security>` for more information. + +Step 2. Define the data model and Datastore +------------------------------------------- + +.. warning:: + The following code snippet will create a new ``products`` table in the database. + Ensure you are using a throwaway schema with no other table named "products" when running this example. + +For this guide, we use the same data model as in the in-memory example. +It manages products in an inventory, so a single collection is sufficient. +Datastores also support managing multiple database tables at the same time if needed. + +Entities map the relational schema to strongly-typed objects that Flows and +Agents can validate at runtime. +The key difference to the in-memory example is that we require the Entity of +interest to be already defined as a table in the database. + +Note that, if you have some columns in the database that are not relevant to your assistant, +you may simply omit them from the Entity definition (as is done in this example with the ``external_system_id``). +However, it may not be possible for the datastore or assistant to create such entities if the omitted columns are required. + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: + :start-after: .. start-##_Schema + :end-before: .. end-##_Schema + + +Step 3. Create datastore steps +------------------------------ + +Now that the Datastore is set up, create steps to perform different operations in the flow. +In this guide, your assistant will identify inconsistencies in product descriptions of the same category. +To do so, you will use the :ref:`DatastoreQueryStep ` to fetch product information, and a :ref:`PromptExecutionStep ` to identify the issues. + +In particular, the ``DatastoreQueryStep`` can be used with Database Datastores to execute developer-defined SQL queries. +These queries can optionally be parametrized with bind variables. +See the `bind variables guide on the python-oracledb documentation `_ for more information. +See also :ref:`the list of all available Datastore steps ` for additional operations that can be performed with Oracle Database Datastores. + + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: + :start-after: .. start-##_Create_Datastore_step_and_flow + :end-before: .. end-##_Create_Datastore_step_and_flow + +Finally, verify that the Flow works as expected: + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: + :start-after: .. start-##_Execute_flow + :end-before: .. end-##_Execute_flow + + +Agent Spec Exporting/Loading +============================ + +This flow can be exported to Agent Spec using the ``AgentSpecExporter`` as you have seen in the +previous in-memory example, using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_connect_to_oracle_database.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_connect_to_oracle_database.yaml + :language: yaml + + +.. warning:: + + The Oracle Database Connection Config objects contain several sensitive values + (like username, password, wallet location) that will not be serialized by the ``AgentSpecExporter``. + These will be serialized as references that must be resolved at loading time, by specifying the values + of these sensitive fields in the ``component_registry`` argument of the loader: + + .. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :start-after: .. start-##_Provide_sensitive_information_when_loading_the_Agent_Spec_config + :end-before: .. end-##_Provide_sensitive_information_when_loading_the_Agent_Spec_config + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_connect_to_oracle_database.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginDatastoreQueryNode`` + - ``PluginOracleDatabaseDatastore`` + - ``PluginTlsOracleDatabaseConnectionConfig`` + - ``PluginOutputMessageNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Having learned how to connect WayFlow assistants to data sources, you may now proceed to: + +- :doc:`Create Conditional Transitions in Flows ` +- :doc:`Create a ServerTool from a Flow ` + + +Full code +========= + +In-memory datastore +------------------- + +Click on the card at the :ref:`top of this page ` +to download the full code for this guide or copy the code below. + +.. collapse:: Connecting Assistants to User Data (full code) + + .. literalinclude:: ../end_to_end_code_examples/howto_datastores.py + :language: python + :linenos: + +Oracle Database datastore +------------------------- + +Click on the card at the :ref:`top of this page ` +to download the full code for this guide, or copy the code below. + +.. collapse:: Connecting Assistants to Oracle Database (full code) + + .. literalinclude:: ../end_to_end_code_examples/howto_connect_to_oracle_database.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_evaluation.rst.txt b/26.1.2/_sources/core/howtoguides/howto_evaluation.rst.txt new file mode 100644 index 000000000..7fa9eb2e9 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_evaluation.rst.txt @@ -0,0 +1,106 @@ +.. _top-howtoevaluation: + +========================== +How to Evaluate Assistants +========================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_evaluation.py + :link-alt: Evaluate Assistants how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`WayFlow Agents <../tutorials/basic_agent>` + +Evaluating the robustness and performance of assistants requires careful, reproducible measurement. You can benchmark assistants on a dataset and report metrics. +This is what the :ref:`AssistantEvaluator ` is designed for. + +The ``AssistantEvaluator`` works as follows: + +.. image:: ../_static/howto/assistant_evaluator.png + :align: center + :scale: 40% + +Evaluation is performed by running an :ref:`AssistantEvaluator ` over a set of :ref:`EvaluationTask ` instances within an +:ref:`EvaluationEnvironment `. The environment provides the assistant under test, a human proxy (if needed), and optional lifecycle hooks +(init/reset). Metrics are produced by :ref:`TaskScorer ` implementations attached to the tasks. + +WayFlow supports several LLM API providers. Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Basic implementation +==================== + +A typical end-to-end evaluation includes: + +1. Defining an evaluation environment that supplies the assistant and (optionally) a human proxy. +2. Implementing one or more task scorers to compute metrics. +3. Preparing a set of evaluation tasks (dataset). +4. Running the evaluator and collecting results. + +Define the evaluation environment: + +.. literalinclude:: ../code_examples/howto_evaluation.py + :language: python + :start-after: .. start-##_Define_the_environment + :end-before: .. end-##_Define_the_environment + +Create a task scorer to compute metrics from the assistant conversation: + +.. literalinclude:: ../code_examples/howto_evaluation.py + :language: python + :start-after: .. start-##_Define_the_scorer + :end-before: .. end-##_Define_the_scorer + +Prepare the evaluation configuration (dataset and tasks): + +.. literalinclude:: ../code_examples/howto_evaluation.py + :language: python + :start-after: .. start-##_Define_the_evaluation_config + :end-before: .. end-##_Define_the_evaluation_config + +Run the evaluation and inspect the results: + +.. literalinclude:: ../code_examples/howto_evaluation.py + :language: python + :start-after: .. start-##_Run_the_evaluation + :end-before: .. end-##_Run_the_evaluation + +.. hint:: + **Task kwargs** vs **Scoring kwargs** + + - Use task kwargs to parameterize task execution (information the assistant needs). + - Use scoring kwargs to store ground truth and other scoring parameters. + +.. important:: + Task scorers must extend ``TaskScorer`` and follow its API. See the API docs for details. + + +Next steps +========== + +Having learned how to evaluate WayFlow Assistants end-to-end, you can proceed to: + +- :doc:`How to Create Conditional Transitions in Flows ` to branch out depending on the agent's response. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_evaluation.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_event_system.rst.txt b/26.1.2/_sources/core/howtoguides/howto_event_system.rst.txt new file mode 100644 index 000000000..0143d9564 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_event_system.rst.txt @@ -0,0 +1,209 @@ +.. _top-event-system: + +=========================== +How to Use the Event System +=========================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../code_examples/howto_event_system.py + :link-alt: Event System how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Using agents ` + +The event system in WayFlow provides a powerful framework for monitoring and debugging agents and flows. +By capturing detailed runtime data through structured events, it offers deep insights into interactions between agents, flows, tools, and LLMs. + +This guide introduces the core concepts of the event system, describes available event types and listeners, and provides practical examples for effective implementation. + +At its heart, WayFlow's event system records and communicates key occurrences during an execution. Each event is a structured data object that captures details of a specific action or state change, such as starting a conversation, executing a tool, or generating an LLM response. Events include metadata like unique identifiers, timestamps, and relevant contextual information. + +The system follows a publish-subscribe model: events are published as they occur, and components called listeners subscribe to receive and react to them. This separation of event generation and handling allows developers to add custom behaviors or logging without altering the core logic of agents or flows. + +Basic Implementation +==================== + +To use the event system effectively, you need to understand its two main components: :ref:`Events ` and :ref:`EventListeners `. + +- :ref:`Events ` are data structures that represent occurrences within WayFlow, organized into different types for various scenarios. +- :ref:`EventListeners ` are components that react to these published events. + +Let's explore this with two practical examples: + +Example 1: Computing LLM Token Usage +==================================== + +A key use of the event system is tracking resource consumption, such as monitoring token usage during LLM interactions. +Since token usage affects operational costs, this data can inform prompt and model optimization. +By subscribing to LLM response events, developers can aggregate and analyze token usage across a conversation. + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_TokenUsage + :end-before: .. end-##_TokenUsage + +In this example, ``TokenUsageListener`` is a custom listener that calculates total token usage by summing the tokens reported in each :ref:`LlmGenerationResponseEvent `. + +Example 2: Tracking Tool Calls +============================== + +Another useful application is monitoring tool invocations within an agentic workflow. +Understanding which tools are used and their frequency helps developers evaluate the effectiveness of their toolset and identify opportunities for improvement. +By listening to tool execution events, you can log each call and track usage patterns. + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_Tool_Call_Listener + :end-before: .. end-##_Tool_Call_Listener + +This snippet illustrates how to create a ``ToolCallListener`` to track tool invocations using :ref:`ToolExecutionStartEvent `. + +With both listeners implemented, let's apply them in a conversation with an :ref:`Agent `. + +For LLMs, WayFlow supports multiple API providers. Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Now, let's set up an agent: + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_Agent + :end-before: .. end-##_Agent + +Using the agent in a conversation: + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_Conversation + :end-before: .. end-##_Conversation + +Both listeners are registered within a context manager using :ref:`register_event_listeners ` during agent execution, ensuring they capture all relevant events. + +Beyond the events highlighted here, WayFlow offers a wide range of events for detailed monitoring. Below is a table explaining various types of events in Wayflow: + +.. _listofsupportedevents: +.. list-table:: WayFlow Event Types + :header-rows: 1 + :widths: 30 70 + + * - Event Name + - Description + * - :ref:`Event ` + - Base event class containing information relevant to all events. + * - :ref:`LlmGenerationRequestEvent ` + - Recorded when the LLM receives a generation request. + * - :ref:`LlmGenerationResponseEvent ` + - Recorded when the LLM generates a response. + * - :ref:`ConversationalComponentExecutionStartedEvent ` + - Recorded when the agent/flow execution has started. + * - :ref:`ConversationalComponentExecutionFinishedEvent ` + - Recorded when the agent/flow execution has ended. + * - :ref:`ConversationCreatedEvent ` + - Recorded whenever a new conversation with an agent or a flow was created. + * - :ref:`ConversationMessageAddedEvent ` + - Recorded whenever a new message was added to the conversation. + * - :ref:`ConversationMessageStreamStartedEvent ` + - Recorded whenever a new message starts being streamed to the conversation. + * - :ref:`ConversationMessageStreamChunkEvent ` + - Recorded whenever a message is being streamed and a delta is added to the conversation. + * - :ref:`ConversationMessageStreamEndedEvent ` + - Recorded whenever a streamed message to the conversation ends. + * - :ref:`ConversationExecutionStartedEvent ` + - Recorded whenever a conversation is started. + * - :ref:`ConversationExecutionFinishedEvent ` + - Recorded whenever a conversation execution finishes. + * - :ref:`ToolExecutionStartEvent ` + - Recorded whenever a tool is executed. + * - :ref:`ToolExecutionResultEvent ` + - Recorded whenever a tool has finished execution. + * - :ref:`ToolConfirmationRequestStartEvent ` + - Recorded whenever a tool confirmation is required. + * - :ref:`ToolConfirmationRequestEndEvent ` + - Recorded whenever a tool confirmation has been handled. + * - :ref:`StepInvocationStartEvent ` + - Recorded whenever a step is invoked. + * - :ref:`StepInvocationResultEvent ` + - Recorded whenever a step invocation has finished. + * - :ref:`ContextProviderExecutionRequestEvent ` + - Recorded whenever a context provider is called. + * - :ref:`ContextProviderExecutionResultEvent ` + - Recorded whenever a context provider has returned a result. + * - :ref:`FlowExecutionIterationStartedEvent ` + - Recorded whenever an iteration of a flow has started executing. + * - :ref:`FlowExecutionIterationFinishedEvent ` + - Recorded whenever an iteration of a flow has finished executing. + * - :ref:`AgentExecutionIterationStartedEvent ` + - Recorded whenever an iteration of an agent has started executing. + * - :ref:`AgentExecutionIterationFinishedEvent ` + - Recorded whenever an iteration of an agent has finished executing. + * - :ref:`ExceptionRaisedEvent ` + - Recorded whenever an exception occurs. + * - :ref:`AgentNextActionDecisionStartEvent ` + - Recorded at the start of the agent taking a decision on what to do next. + * - :ref:`AgentDecidedNextActionEvent ` + - Recorded whenever the agent decided what to do next. + +See :ref:`Events ` for more information. You can implement custom ``EventListener`` for these events as shown in the examples above. + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_event_system.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_event_system.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_event_system.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +Next Steps +========== + +After exploring the event system in WayFlow, consider learning more about related features to further enhance your agentic applications: + +- :doc:`How to Enable Tracing in WayFlow ` +- :doc:`How to Build a Swarm of Agents ` + +Full Code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_event_system.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_execute_agentspec_with_wayflowcore.rst.txt b/26.1.2/_sources/core/howtoguides/howto_execute_agentspec_with_wayflowcore.rst.txt new file mode 100644 index 000000000..98675a051 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_execute_agentspec_with_wayflowcore.rst.txt @@ -0,0 +1,150 @@ +.. _top-execute-agentspec: + +============================================= +Execute Agent Spec Configuration with WayFlow +============================================= + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_execute_agentspec_with_wayflowcore.py + :link-alt: Execute Agent Spec with WayFlow how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with `Agent Spec `_ and exporting Agent Spec configurations. + + +This guide will show you how to: + +1. Define the execution of tools in a tool registry. +2. Load an Agent Spec configuration with the WayFlow adapter. +3. Run the assistant and interact with it. + +This guide shows you how to use a minimal Agent setup with a single tool for performing multiplications. +You can use the same code structure to load and run more complex assistants. + + +1. Install packages +------------------- + +To run the examples in this guide, make sure that ``wayflowcore`` is installed. + +For more details, refer to the `WayFlow Agent Spec adapter API documentation `_. + +2. Define the tool registry +--------------------------- + +Before loading the configuration, you need to define the tool registry. +This registry specifies how to execute each tool since the implementation of tools is not +included in the Agent Spec assistant configuration. + +The example below shows how to register a single tool that performs multiplications. +Adapt this pattern to register all the tools your assistant requires. + +This guide uses the simplest type of tool, ``ServerTool``. +For more advanced tool types, refer to the `Agent Spec Language specification `_. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: + :start-after: .. start-##_Tool_Registry_Setup + :end-before: .. end-##_Tool_Registry_Setup + + +3. Load the Agent Spec configuration +------------------------------------ + +Now load the agent configuration. +The configuration starts by defining two components: ``multiplication_tool`` and ``vllm_config``. +These are referenced in the agent definition later. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: + :start-after: .. start-##_AgentSpec_Configuration + :end-before: .. end-##_AgentSpec_Configuration + +Loading the configuration to the WayFlow executor is simple as long as the ``tool_registry`` has been defined. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: + :start-after: .. start-##_Load_AgentSpec_Configuration + :end-before: .. end-##_Load_AgentSpec_Configuration + + +4. Run the assistant +-------------------- + +Start by creating a conversation. +Then prompt the user for some input. +The assistant will respond until you interrupt the process with Ctrl+C. +Messages of type ``TOOL_REQUEST`` and ``TOOL_RESULT`` are also displayed. +These messages help you understand how the assistant decides to act. + +For more details, see the :doc:`API Reference <../api/conversation>`. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: + :start-after: .. start-##_Execution_Loop_Conversational + :end-before: .. end-##_Execution_Loop_Conversational + +You can also run a non-conversational flow. +In that case, the execution loop could be implemented as follows: + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: + :start-after: .. start-##_Execution_Loop_Non_Conversational + :end-before: .. end-##_Execution_Loop_Non_Conversational + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + +And load it back using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + + +Next steps +========== + +In this guide, you learned how to: + +1. Install the Agent Spec adapter for WayFlow. +2. Define tool execution using a tool registry. +3. Load an Agent Spec configuration with the WayFlow adapter. +4. Run the assistant and interact with it in a conversation loop. + +You may now proceed to: + +- :doc:`How to Connect to a MCP Server ` +- :doc:`How to Add User Confirmation to Tool Call Requests ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_execute_agentspec_with_wayflowcore.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_flowbuilder.rst.txt b/26.1.2/_sources/core/howtoguides/howto_flowbuilder.rst.txt new file mode 100644 index 000000000..15c687050 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_flowbuilder.rst.txt @@ -0,0 +1,117 @@ +===================================== +Build Flows with the Flow Builder +===================================== + +This guide shows how to assemble WayFlow Flows using the chainable ``FlowBuilder`` API. + +.. admonition:: Prerequisites + + This guide assumes you are familiar with the following concepts: + + - :ref:`Flows ` and basic steps/edges + + +Overview +======== + +``FlowBuilder`` lets you quickly construct flows without manually wiring every edge. It supports: + +- ``add_sequence``: add steps in order and wire control edges between them. +- ``set_entry_point`` and ``set_finish_points``: declare the entry and terminal steps. +- ``add_conditional``: branch based on an input to a :ref:`BranchingStep `. +- ``build_linear_flow``: convenience to assemble a linear flow in one call. + +See the full API in :doc:`API › Flows <../api/flows>` and quick snippets in the :ref:`Reference Sheet `. + + +1. Build a linear flow +====================== + +Create two steps and connect them linearly with a single call. + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_linear_flow + :end-before: .. end-##_Build_a_linear_flow + +API Reference: :ref:`FlowBuilder ` + + +You can also use the ``build_linear_flow`` method: + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_linear_flow_equivalent + :end-before: .. end-##_Build_a_linear_flow_equivalent + + +2. Add a conditional branch +=========================== + +Add a branching step where an upstream step’s output determines which branch to execute. + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_flow_with_a_conditional + :end-before: .. end-##_Build_a_flow_with_a_conditional + +Notes: + +- ``add_conditional`` accepts the branch key as a string output name, or a tuple ``(step_or_name, output_name)`` to read from another step. +- ``set_finish_points`` declares which steps finish the flow (creates control edges to ``CompleteStep``). + + +3. Export the flow +================== + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + + +Here is what the Agent Spec representation will look like ↓ +----------------------------------------------------------- + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_flowbuilder.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_flowbuilder.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Recap +===== + +This how-to guide showed how to: + +- Build a linear flow in one line with ``build_linear_flow`` +- Add a conditional branch with ``set_conditional`` +- Declare entry and finish points and serialize your flow + + +Next steps +========== + +- Explore more patterns in the :ref:`Reference Sheet ` +- See the complete API in :doc:`API › Flows <../api/flows>` +- Learn about branching and loops in :doc:`How to Develop a Flow with Conditional Branches ` diff --git a/26.1.2/_sources/core/howtoguides/howto_imagecontent.rst.txt b/26.1.2/_sources/core/howtoguides/howto_imagecontent.rst.txt new file mode 100644 index 000000000..a8146f789 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_imagecontent.rst.txt @@ -0,0 +1,173 @@ +.. _top-howtoimagecontent: + +===================================== +How to Send Images to LLMs and Agents +===================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_imagecontent.py + :link-alt: Tutorial on using Images with WayFlow models + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + - Familiarity with :doc:`basic agent and prompt workflows <../tutorials/basic_agent>`. + +Overview +======== + +Some Large Language Models (LLMs) can handle images in addition to text. +WayFlow supports passing images alongside text in both direct prompt requests and full agent conversations +using the `ImageContent` API. + +This guide will show you: + +- How to create `ImageContent` in code. +- How to run a prompt with image input directly with the model. +- How to send image+text messages in an Agent conversation. +- How to inspect and use model/agent outputs with image reasoning. + +What is ``ImageContent``? +------------------------- + +`ImageContent` is a type of message content that stores image bytes and format metadata. +You can combine an image with additional `TextContent` in a single message. + + +Basic implementation +==================== + +First import what is needed for this guide: + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :linenos: + :start-after: .. start-##_Imports + :end-before: .. end-##_Imports + +To follow this guide, you will need access to a **Multimodal** large language model (LLM). +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Step 1: Creating a prompt with ImageContent +=========================================== + +Before sending requests to your vision-capable LLMs or agents, you need to construct a prompt containing both the image and text content. The example below demonstrates: + +- Downloading an image (here, the Oracle logo) via HTTP request +- Creating an `ImageContent` object from the image bytes +- Adding a `TextContent` question +- Packing both into a `Message`, then into a `Prompt` + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :start-after: .. start-##_Create_prompt + :end-before: .. end-##_Create_prompt + +Step 2: Sending image input to a vision-capable model +===================================================== + +You can send images directly to your LLM by constructing a prompt with both `ImageContent` and `TextContent`. +The example below downloads the Oracle logo PNG and queries the LLM for recognition. + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :start-after: .. start-##_Generate_completion_with_an_image_as_input + :end-before: .. end-##_Generate_completion_with_an_image_as_input + + +**Expected output:** The model should identify the company (e.g. "Oracle Corporation" or equivalent). +If your model does not support images, you will get an error. + +Step 3: Using images in Agent conversations +=========================================== + +You can pass images in an Agent-driven chat workflow. +This allows assistants to process visual information alongside user dialog. + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :start-after: .. start-##_Pass_an_image_to_an_agent_as_input + :end-before: .. end-##_Pass_an_image_to_an_agent_as_input + +**Expected output:** The agent response should mention "Oracle Corporation". + +API Reference and Practical Information +======================================= + +- :ref:`ImageContent ` +- :ref:`TextContent ` +- :class:`wayflowcore.agent.Agent` + +Supported Image Formats +----------------------- + +Most vision LLMs support PNG, JPG, JPEG, GIF, or WEBP. +Always specify the correct format for ImageContent. + + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_imagecontent.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_imagecontent.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_imagecontent.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Next steps +========== + +Having learned how to send images to LLMs and Agents, you may now proceed to: + +- :doc:`../tutorials/basic_agent` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_imagecontent.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_long_context.rst.txt b/26.1.2/_sources/core/howtoguides/howto_long_context.rst.txt new file mode 100644 index 000000000..72c7b7422 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_long_context.rst.txt @@ -0,0 +1,210 @@ +.. _top-howtolongmessage: + +============================================ +How to Enable Agents to Handle Long Contexts +============================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_long_context.py + :link-alt: Long message how-to script + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLM configuration <../howtoguides/llm_from_different_providers>` + - :doc:`Using agents ` + - :doc:`Advanced Prompting Techniques ` + + +In agentic systems, conversations can become lengthy, building up extensive context over many interactions. +Large Language Models (LLMs) have limits on the context they can process effectively, which may result in reduced +performance or errors when these limits are surpassed. Long context also incur higher costs. + +To address these performance issues and reduce cost, you can apply techniques that reduce the context size while retaining key information. + +This guide demonstrates three methods for reducing context size using built-in transforms: + +- **Summarizing long messages** using :ref:`MessageSummarizationTransform `: Individual messages that exceed size limits (such as large tool outputs or long user inputs) are automatically summarized using an LLM. +- **Summarizing long conversations** using :ref:`ConversationSummarizationTransform `: When conversations become too lengthy, older messages are summarized to maintain context while keeping the total size manageable. +- **Using both transforms together**: Combining both approaches for comprehensive context management. + + +This guide shows how to use built-in :ref:`MessageTransform ` objects to reduce the context size in agents. +The same transforms can also be attached directly to :ref:`Swarm ` and :ref:`ManagerWorkers ` +through their ``transforms`` constructor argument when you need long-context handling in multi-agent patterns. + +Introduction +============ + +Message Transforms +~~~~~~~~~~~~~~~~~~ + +A :ref:`MessageTransform ` is a transformation applied to a list of messages before they are sent to the LLM powering the agent. You can learn +more about them in the :doc:`Advanced Prompting Techniques ` guide. + +In this guide, we use built-in message transforms to adjust the agent's chat history by passing them directly to the :ref:`Agent ` constructor. +For :ref:`Swarm ` and :ref:`ManagerWorkers `, the ``transforms`` constructor argument behaves the same way: +component-level transforms are prepended to any transforms already defined on the prompt template, so user-provided transforms run first. +For ``Swarm``, when the active agent also defines ``Agent(..., transforms=[...])``, those agent-level transforms run before +the swarm-level transforms. See :ref:`Swarm transform ordering ` for the full ordering. + +LLM Setup +~~~~~~~~~ + +There are multiple ways to define an LLM in WayFlow depending on the provider. +Choose the one adapted to your needs: + +.. include:: ../_components/llm_config_tabs.rst + +In this guide, we will use `VllmModel`. We can use two different LLMs - a capable LLM as the agent and a fast LLM as the summarizer. Note that if conversations include images, the summarization LLM should support image processing: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Define_LLMS + :end-before: .. end-##_Define_LLMS + + + +Summarizing Long Messages +========================= + +We use the built-in :ref:`MessageSummarizationTransform ` to automatically summarize individual messages that exceed a specified size limit, including tool outputs, user messages and images. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_MessageSummarizationTransform + :end-before: .. end-##_Create_MessageSummarizationTransform + +.. note:: + When creating :ref:`MessageSummarizationTransform ` without specifying a ``datastore``, it will initialize a default :ref:`InMemoryDatastore ` for caching, which is only suitable for prototyping. This will raise a user warning indicating that in-memory datastores are not recommended for production systems. + For production systems, you can use :ref:`OracleDatabaseDatastore ` or :ref:`PostgresDatabaseDatastore `. + +.. important:: + If you serialize and later deserialize a conversation or prompt template that uses summarization transforms, the transform configuration is restored, including the summarization LLM settings and datastore configuration. + However, an :ref:`InMemoryDatastore ` does not preserve cached summary rows across process restarts. If you need summary cache reuse after deserialization in a persistent application, use a database-backed datastore such as :ref:`OracleDatabaseDatastore ` or :ref:`PostgresDatabaseDatastore `. + +Let's integrate the :ref:`MessageSummarizationTransform ` into the :ref:`Agent `: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_Agent + :end-before: .. end-##_Create_Agent + +Before seeing an example, let's define a token counting event listener that will help us see the impact of our summarizer on token consumption. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_Token_Event_Listener + :end-before: .. end-##_Create_Token_Event_Listener + +In this example, we run a conversation where the agent uses a tool that returns a very long log output. The :ref:`MessageSummarizationTransform ` automatically summarizes this long message to stay within size limits. The agent then answers questions about the error, demonstrating that key information is preserved despite the summarization. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Run_Agent_With_MessageSummarizationTransform + :end-before: .. end-##_Run_Agent_With_MessageSummarizationTransform + +To compare, let's run the same conversation without any transforms to see the difference in token usage. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_Agent_Without_Transform + :end-before: .. end-##_Create_Agent_Without_Transform + +After one user message, we have used the following number of tokens: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Run_Agent_Without_Transform + :end-before: .. end-##_Run_Agent_Without_Transform + +And after the second user message: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Run_Agent_Without_Transform_Round2 + :end-before: .. end-##_Run_Agent_Without_Transform_Round2 + +- **After one user message**, both approaches show high token usage: + + - Without :ref:`MessageSummarizationTransform `: the agent LLM must process the full long tool output (around 12,800 tokens) + + - With :ref:`MessageSummarizationTransform `: the summarization LLM generates a summary (around 8,900 tokens) + +- **After a second message**, the difference becomes significant: + + - Without the transform: token usage doubles to about 25,300 tokens as the agent LLM must reprocess the long tool output + + - With the transform: token usage only increases slightly to about 9,400 tokens due to summaries being cached in datastores and not regenerated. + +This demonstrates that the messages were effectively summarized and cached. + +Summarizing Long Conversations +============================== + +This method uses the built-in :ref:`ConversationSummarizationTransform ` to summarize older messages when conversations become too long, preserving historical information while keeping the context manageable. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_ConversationSummarizationTransform + :end-before: .. end-##_Create_ConversationSummarizationTransform + +If you prefer a best-effort size threshold instead of a message-count threshold, set ``max_num_messages=None`` and use ``max_num_characters``: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_ConversationSummarizationTransform_With_Character_Threshold + :end-before: .. end-##_Create_ConversationSummarizationTransform_With_Character_Threshold + +Let's integrate the :ref:`MessageTransform ` into the :ref:`Agent `: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Run_Agent_With_ConversationSummarizationTransform + :end-before: .. end-##_Run_Agent_With_ConversationSummarizationTransform + +If you want to check that the :ref:`ConversationSummarizationTransform ` is really summarizing old messages together, you either use the same ``TokenLister`` as in the previous section to measure tokens or you can also create your own ``EventListener`` that prints the conversation. + +Using Both Transforms Together +============================== + +For comprehensive context management, you can apply both transforms together. The :ref:`MessageSummarizationTransform ` will first handle individual long messages, and then the :ref:`ConversationSummarizationTransform ` will manage the overall conversation length. + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Create_Both_Transforms + :end-before: .. end-##_Create_Both_Transforms + +Let’s integrate the :ref:`MessageTransform ` into the :ref:`Agent `: + +.. literalinclude:: ../code_examples/howto_long_context.py + :language: python + :start-after: .. start-##_Run_Agent_With_Both_Transforms + :end-before: .. end-##_Run_Agent_With_Both_Transforms + + +Next Steps +========== + +We have seen in this how-to how to leverage :ref:`MessageSummarizationTransform ` and :ref:`ConversationSummarizationTransform ` to reduce the size of the LLM context. +We have also seen how to use event listeners to measure LLM token consumption. With this new knowledge, you can proceed to :doc:`How to Build Assistants with Tools `. + + +Full Code +========= + +Click the card at the :ref:`top of this page ` to download the complete code for this guide, or copy it below. + +.. literalinclude:: ../end_to_end_code_examples/howto_long_context.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_managerworkers.rst.txt b/26.1.2/_sources/core/howtoguides/howto_managerworkers.rst.txt new file mode 100644 index 000000000..614d6cc07 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_managerworkers.rst.txt @@ -0,0 +1,372 @@ +.. _top-howtomanagerworkers: + +======================================== +How to Build a Manager-Workers of Agents +======================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_managerworkers.py + :link-alt: ManagerWorkers how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Agents <../tutorials/basic_agent>`. + +With the advent of increasingly powerful Large Language Models (LLMs), multi-agent systems are becoming more relevant +and are expected to be particularly valuable in scenarios requiring high-levels of autonomy and/or processing +of diverse sources of information. + +There are various types of multi-agent systems, each serving different purposes and applications. +Some notable examples include hierarchical structures, agent swarms, and mixtures of agents. + +This guide demonstrates an example of a hierarchical multi-agent system (also known as manager-workers pattern) and will show you how to: + +- Build expert agents equipped with tools and a manager agent; +- Test the expert agents individually; +- Build a ManagerWorkers using the defined agents; +- Execute the ManagerWorkers of agents; + +.. image:: ../_static/howto/howto_multiagent.svg + :align: center + :scale: 70% + :alt: Example of a multi-agent system + +**Diagram:** Multi-agent system shown in this how-to guide, comprising a manager agent +(customer service agent) and two expert agents equipped with tools (refund specialist +and satisfaction surveyor). + +.. seealso:: + + To access short code snippets demonstrating how to use other agentic patterns in WayFlow, + refer to the :doc:`Reference Sheet <../misc/reference_sheet>`. + + +To follow this guide, you need an LLM. WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Building and testing expert Agents +================================== + +In this guide you will use the following helper function to print +messages: + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Helper_method_for_printing_conversation_messages + :end-before: .. end-##_Helper_method_for_printing_conversation_messages + +API Reference: :ref:`MessageType ` + +Refund specialist agent +----------------------- + +The refund specialist agent is equipped with two tools. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Specialist_tools + :end-before: .. end-##_Specialist_tools + +API Reference: :ref:`tool ` + +The first tool is used to check whether a given order is eligible for a refund, +while the second is used to process the specific refund. + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Specialist_prompt + :end-before: .. end-##_Specialist_prompt + +.. important:: + + The quality of the system prompt is paramount to ensuring proper behaviour of the multi-agent + system, because slight deviations in the behaviour can lead to cascading + unintended effects as the number of agents scales up. + +Building the Agent +^^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Specialist_agent + :end-before: .. end-##_Specialist_agent + +API Reference: :ref:`Agent ` + +Testing the Agent +^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Specialist_test + :end-before: .. end-##_Specialist_test + +Test the agents individually to ensure they perform as expected. + + +Statisfaction surveyor agent +---------------------------- + +Tools +^^^^^ + +The statisfaction surveyor agent is equipped with one tool. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Surveyor_tools + :end-before: .. end-##_Surveyor_tools + + +The ``record_survey_response`` tool is simulating the recording of +user feedback data. + + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Surveyor_prompt + :end-before: .. end-##_Surveyor_prompt + + +Building the Agent +^^^^^^^^^^^^^^^^^^ + + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Surveyor_agent + :end-before: .. end-##_Surveyor_agent + + +Testing the Agent +^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Surveyor_test + :end-before: .. end-##_Surveyor_test + + +Again, the expert agent behaves as intended. + +Manager Agent +------------- + +In the our built-in ManagerWorkers component, we allow passing an Agent +as the group manager. Therefore, we just need to define an agent as usual. + +In this example, our manager agent will be a Customer Service Manager. + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Manager_prompt + :end-before: .. end-##_Manager_prompt + +Building the manager Agent +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Manager_agent + :end-before: .. end-##_Manager_agent + + +Building and testing ManagerWorkers of Agents +============================================= + +Building the ManagerWorkers of Agents +------------------------------------- + + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Managerworkers_pattern + :end-before: .. end-##_Managerworkers_pattern + +API Reference: :ref:`ManagerWorkers ` + +The ManagerWorkers has two main parameters: + +- ``group_manager`` + This can be either an Agent or an LLM. + + - If an LLM is provided, a manager agent will automatically be created using that LLM along with the default ``custom_instruction`` for group managers. + - In this example, we explicitly pass an Agent (the *Customer Service Manager Agent*) so we can use our own defined ``custom_instruction``. + +- ``workers`` - List of Agents + These agents serve as the workers within the group and are coordinated by the manager agent. + + - Worker agents cannot interact with the end user directly. + - When invoked, each worker can leverage its equipped tools to complete the assigned task and report the result back to the group manager. + +``ManagerWorkers`` also accepts a ``transforms`` parameter. This is the recommended way to apply message transforms such as +:ref:`MessageSummarizationTransform ` or :ref:`ConversationSummarizationTransform ` +to the manager's rendered prompt in a multi-agent setup. +These component-level transforms are prepended to any transforms already present on the ``managerworkers_template``. +If ``group_manager`` is itself an ``Agent`` with its own ``transforms``, those manager-agent transforms are also preserved during execution. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Managerworkers_pattern_with_transforms + :end-before: .. end-##_Managerworkers_pattern_with_transforms + +.. _managerworkers_transform_ordering: + +.. note:: + **How ``transforms`` and ``managerworkers_template`` work together** + + During execution, WayFlow builds the manager agent's effective runtime template from the configured ``managerworkers_template``. + If ``group_manager`` is an ``Agent`` with its own pre-rendering transforms, those run first, then ``managerworkers.transforms``, and finally any pre-rendering transforms already attached directly to ``managerworkers_template``. + The effective order is therefore: + + 1. ``group_manager.transforms`` for the group manager, when ``group_manager`` is an ``Agent`` + 2. ``managerworkers.transforms`` in the order you passed them + 3. ``managerworkers_template.pre_rendering_transforms`` + + This means the most specific transforms run first: manager-agent before manager-workers, and manager-workers before template-level transforms. + +When a ``ManagerWorkers`` conversation is serialized and later deserialized, the summarization transform configuration is restored. +Use a persistent datastore if you also need the summary cache contents to survive application restarts. + +Executing the ManagerWorkers +---------------------------- + +The power of mult-agent systems is their high adaptiveness. +In the following example, it is demonstrated how the manager can decide +not to call the expert agents for simple user queries. + + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Managerworkers_answers_without_expert + :end-before: .. end-##_Managerworkers_answers_without_expert + +However, the manager is explicitly prompted to assign to the specialized agents for more complex tasks. +This is demonstrated in the following example. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Managerworkers_answers_with_expert + :end-before: .. end-##_Managerworkers_answers_with_expert + + +Agent Spec Exporting/Loading +---------------------------- + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_managerworkers.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_managerworkers.yaml + :language: yaml + +You can load it back using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Using ManagerWorkers within a Flow +================================== + +The ``Manager-Workers`` pattern can be integrated into a :ref:`Flow ` using the :ref:`AgentExecutionStep `. + +Here's an example of how to integrate a manager-workers system into a flow: + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Using_ManagerWorkers_within_a_Flow + :end-before: .. end-##_Using_ManagerWorkers_within_a_Flow + +You can run the flow with: + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Run_ManagerWorkers_within_a_Flow + :end-before: .. end-##_Run_ManagerWorkers_within_a_Flow + +Agent Spec Exporting/Loading +---------------------------- + +You can export the flow configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec2 + :end-before: .. end-##_Export_config_to_Agent_Spec2 + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the flow configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_managerworkers2.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_managerworkers2.yaml + :language: yaml + +You can then load the configuration back to a flow using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_managerworkers.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config2 + :end-before: .. end-##_Load_Agent_Spec_config2 + + +Next steps +========== + +Now that you have learned how to define a ManagerWorkers, you may proceed to :doc:`Build a Swarm of Agents `. + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_managerworkers.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_mapstep.rst.txt b/26.1.2/_sources/core/howtoguides/howto_mapstep.rst.txt new file mode 100644 index 000000000..62059eae7 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_mapstep.rst.txt @@ -0,0 +1,210 @@ +.. _top-howtomapstep: + +============================================ +How to Do Map and Reduce Operations in Flows +============================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_mapstep.py + :link-alt: MapStep how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +Map-Reduce is a programming model essential for efficiently processing large datasets across distributed systems. +It is widely used in software engineering to enhance data processing speed and scalability by parallelizing tasks. + +WayFlow supports the Map and Reduce operations in Flows, using the :ref:`MapStep `. +This guide will show you how to: + +- use :ref:`MapStep ` perform an operation on **all elements of a list** +- use :ref:`MapStep ` to perform an operation on **all key/value pairs of a dictionary** +- use :ref:`MapStep ` to **parallelize** some operations + +.. image:: ../_static/howto/mapstep.svg + :align: center + :scale: 100% + :alt: Flow diagram of a MapStep + +To follow this guide, you need an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Basic implementation +==================== + +Assuming you want to summarize a few articles. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Define_the_articles + :end-before: .. end-##_Define_the_articles + +You have the option to generate the summary with the :ref:`PromptExecutionStep ` class, as explained already in :doc:`the separate guide `: + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Create_the_Flow_for_the_MapStep + :end-before: .. end-##_Create_the_Flow_for_the_MapStep + +This step takes a single article, and generates a summary. +Since you have a list of articles, use the ``MapStep`` class to generate a summary for each article. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Create_the_MapStep + :end-before: .. end-##_Create_the_MapStep + +.. note:: + + In the ``unpack_input`` function, define how each sub-flow input is retrieved. + Here, the sub-flow requires an ``article`` input. Set its value to ``.``, because each iterated item is the article and ``.`` is the identity + query in JQ. + + The ``output_descriptors`` parameter specifies which outputs of the sub-flow will be collected and merged into a list. + +Once this is done, create the flow for the ``MapStep`` and execute it: + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Create_and_execute_the_final_Flow + :end-before: .. end-##_Create_and_execute_the_final_Flow + +As expected, your flow has generated summaries of three articles! + + +Processing in parallel +====================== + +By default, the :ref:`MapStep ` runs all operations sequentially in order. +This is done so that any flow (including flows that yield or ask the user) can be run. + +In many cases (such as generating articles summary), the work is completely parallelizable because the operations are independent from each other. +In this context, you can just set the ``parallel_execution`` parameter to ``True`` and the operations will be run in parallel using a thread-pool. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Parallel_execution_of_map_reduce_operation + :end-before: .. end-##_Parallel_execution_of_map_reduce_operation + +The same can be achieved using the :ref:`ParallelMapStep `. +This step type is equivalent to the :ref:`MapStep `, the only difference is that parallelization is always enabled. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Parallel_execution_of_map_reduce_operation_with_ParallelMapStep + :end-before: .. end-##_Parallel_execution_of_map_reduce_operation_with_ParallelMapStep + +.. note:: + The Global Interpreter Lock (GIL) in Python is not a problem for parallel remote requests + because I/O-bound operations, such as network requests, release the GIL during their execution, + allowing other threads to run concurrently while the I/O operation is in progress. + +Not all sub-flows can be executed in parallel. +The table below summarizes the limitations of parallel execution for the :ref:`MapStep `: + + .. list-table:: + :widths: 30 50 50 45 + :header-rows: 1 + + * - Support + - Type of flow + - Examples + - Remarks + * - **FULLY SUPPORTED** + - **Flows that do not yield and do not have any side-effect on the conversation** (no variable read/write, posting to the conversation, and so on) + - Embarrassingly parallel flows (simple independent operation), such as a ``PromptExecutionStep``, ``ApiCallStep`` to post or get, and so on + - N/A + * - **SUPPORTED WITH SIDE EFFECTS** + - **Flows that do not yield but have some side-effect on the conversation** (variable read/write, posting to the conversation, and so on) + - Flows with ``OutputMessageStep``, ``VariableStep``, ``VariableReadStep``, ``VariableWriteStep``, and so on + - No guarantee in the order of operations (such as posting to the conversation), only the outputs are guaranteed in order. + * - **NON SUPPORTED** + - **Flows that yield**. WayFlow does not support this, otherwise a user might be confused in what branch they are currently when prompted to answer. + - Flows with ``InputMessageStep``, ``AgentExecutionStep`` that can ask questions, and so on + - It will raise an exception at instantiation time if a sub-flow can yield and step set to parallel + + +Common patterns and best practices +================================== + +Sometimes, you might have a dictionary, and you need to iterate on each of the key/value pairs. +To achieve this, set ``iterated_input_type`` to ``DictProperty()``, and use the queries ``._key`` (respectively ``._value``) to access the key (and respectively the value) from the key/value pair. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Iterate_over_a_dictionary + :end-before: .. end-##_Iterate_over_a_dictionary + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_mapstep.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_mapstep.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_mapstep.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``ExtendedMapNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Having learned how to perform ``map`` and ``reduce`` operations in WayFlow, you may now proceed to :doc:`How to Use Agents in Flows `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code +for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_mapstep.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_mcp.rst.txt b/26.1.2/_sources/core/howtoguides/howto_mcp.rst.txt new file mode 100644 index 000000000..19b02ded0 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_mcp.rst.txt @@ -0,0 +1,461 @@ +.. _top-howtomcp: + +====================================== +How to connect MCP tools to Assistants +====================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_mcp.py + :link-alt: MCP how-to script + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Tools <../api/tools>` + - :doc:`Building Assistants with Tools ` + +`Model Context Protocol `_ (MCP) is an open protocol that standardizes how applications provide context to LLMs. +You can use an MCP server to provide a consistent tool interface to your agents and flows, without having to create custom adapters for different APIs. + +.. tip:: + + See the `Oracle MCP Server Repository `_ to explore examples + of reference implementations of MCP servers for managing and interacting with Oracle products. + + +In this guide, you will learn how to: + +* Create a simple MCP Server (in a separate Python file) +* Connect an Agent to an MCP Server (including how to export/load via Agent Spec, and run it) +* Connect a Flow to an MCP Server (including export/load/run) + +.. important:: + + This guide does not aim at explaining how to make secure MCP servers, but instead mainly aims at showing how to connect to one. + You should ensure that your MCP server configurations are secure, and only connect to trusted external MCP servers. + + +Prerequisite: Set up a simple MCP Server +======================================== + +First, let’s see how to create and start a simple MCP server exposing a couple of tools. + +.. note:: + You should copy the following server code and run it in a separate Python process. + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##Create_a_MCP_Server + :end-before: # .. end-##Create_a_MCP_Server + +This MCP server exposes two example tools: ``get_user_session`` and ``get_payslips``. +Once started, it will be available at (by default): ``http://localhost:8080/sse``. + + +.. note:: + When choosing a transport for MCP: + + - Use :ref:`Stdio ` when launching and communicating with an MCP server as a local subprocess on the same machine as the client. + - Use :ref:`Streamable HTTP ` when connecting to a remote MCP server. + + For more information, visit https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#stdio + + +Connecting an Agent to the MCP Server +===================================== + +You can now connect an agent to this running MCP server. + + +Add imports and configure an LLM +-------------------------------- + +Start by importing the necessary packages for this guide: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: .. start-##_Imports_for_this_guide + :end-before: .. end-##_Imports_for_this_guide + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Build the Agent +--------------- + +:ref:`Agents ` can connect to MCP tools by either using a :ref:`MCPToolBox ` or a :ref:`MCPTool `. +Here you will use the toolbox (see the section on Flows to see how to use the ``MCPTool``). + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Connecting_an_agent_to_the_MCP_server + :end-before: # .. end-##_Connecting_an_agent_to_the_MCP_server + +Specify the :ref:`transport ` to use to handle the connection to the server and create the toolbox. +You can then equip an agent with the toolbox similarly to tools. + +.. note:: + ``authless_mcp_enabled()`` disables authorization for local/testing only—do not use in production. + Keep it scoped around the code that creates MCP tools or toolboxes. + + +Running the Agent +----------------- + +You can now run the agent in a simple conversation: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Running_the_agent + :end-before: # .. end-##_Running_the_agent + +Alternatively, run the agent interactively in a command-line loop: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Running_with_an_execution_loop + :end-before: # .. end-##_Running_with_an_execution_loop + +.. note:: + + WayFlow maintains MCP Client sessions between calls, which means that the client + does not need to re-authenticate at every call. After establishing a secure connection, + MCP servers can safely perform session recognition (e.g. for retrieving user information) + + +Connecting a Flow to the MCP Server +=================================== + +You can also use MCP tools in a :ref:`Flow ` by using the :ref:`MCPTool ` in a :ref:`ToolExecutionStep `. + +Build the Flow +-------------- + +Create the flow using the MCP tool: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Connecting_a_flow_to_the_MCP_server + :end-before: # .. end-##_Connecting_a_flow_to_the_MCP_server + +Here you specify the client transport as with the MCP ToolBox, as well as the name of the specific tool +you want to use. Additionally, you can override the tool description (exposed by the MCP server) by +specifying the ``description`` parameter. + +.. tip:: + + Use the ``_validate_tool_exist_on_server`` parameter to validate whether the tool is available or not + at instantiation time. + +Running the Flow +---------------- + +Execute the flow as follows: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Running_the_flow + :end-before: # .. end-##_Running_the_flow + + + +Advanced use: Use OAuth in MCP Tools +==================================== + +MCP Tools and ToolBoxes support auth using the official +`OAuth flow from MCP `_, + +To enable auth simply provide an Auth configuration to an MCP Client Transport (SSE or StreamableHTTP). + +.. code-block:: python + + import webbrowser + from wayflowcore.mcp import MCPOAuthConfigFactory + + oauth_callback_port = 8001 # depends on your MCP server configuration + auth = MCPOAuthConfigFactory.with_dynamic_discovery( + redirect_uri=f"http://localhost:{oauth_callback_port}/callback" + ) + client_transport = SSETransport(url=sse_mcp_server_oauth, auth=auth) + + tool = MCPTool( + name="generate_random_string", + description="1234567", + client_transport=client_transport, + input_descriptors=[], + # ^ make sure to specify the input/output descriptors for the tool + ) + + agent = Agent(llm=llm, tools=[tool]) + +.. important:: + + You must disable MCPTool verification at instantiation when using OAuth. + +Then when running the assistant, when authorization is required an execution status +is returned with the authorization URL. The client is responsible for obtaining the +auth code and state and submit it back to the execution loop, which will complete the +OAuth flow. Once the OAuth flow is completed, the conversation can be resumed with the +now authenticated MCP Client sessions. + +.. code-block:: python + + from wayflowcore.auth.auth import AuthChallengeResult + from wayflowcore.executors.executionstatus import AuthChallengeRequestStatus + + conv = agent.start_conversation() + conv.append_user_message("Call the tool please") + status = conv.execute() + + assert isinstance(status, AuthChallengeRequestStatus) + authorization_url = status.auth_request.authorization_url + + # The client app must consume the authorization URL, fetch the code/state + # and submit it back to complete the OAuth flow. + webbrowser.open(authorization_url) + auth_code, auth_state = ... + + # The auth challenge result is submitted, which completes the auth flow + status.submit_result(AuthChallengeResult(code=auth_code, state=auth_state)) + + # The conversation is resumed, and returns the expected result + status = conv.execute() + + +**API Reference:** :ref:`OAuthClientConfig ` + +OAuth works with MCP Tools and ToolBoxes, in agents, flows and multi-agent patterns. + +.. note:: + + Note that MCP client sessions are reused in a single conversation, which means that you + will not have to re-perform the OAuth flow at every request. + + +Advanced use: Complex types in MCP tools +======================================== + + +WayFlow supports MCP tools with non-string outputs, such as: + +- List of string +- Dictionary with keys and values of string type + +From the MCP server-side, you may need to enable the ``structured_output`` parameter +of your MCP server (depending on the implementation). + + +.. code-block:: python + + server = FastMCP( + name="Example MCP Server", + instructions="A MCP Server.", + host=host, + port=port, + ) + + @server.tool(description="Tool that generates a dictionary", structured_output=True) + def generate_dict() -> dict[str, str]: + return {"key": "value"} + + @server.tool(description="Tool that generates a list", structured_output=True) + def generate_list() -> list[str]: + return ["value1", "value2"] + + +On the WayFlow side, the input and output descriptors can be automatically inferred. + +.. code-block:: python + + generate_dict_tool = MCPTool( + name="generate_dict", + description="Tool that generates a dictionary", + client_transport=mcp_client, + # output_descriptors=[DictProperty(name="generate_dictOutput")], # this will be automatically inferred + ) + + generate_list_tool = MCPTool( + name="generate_list", + description="Tool that generates a list", + client_transport=mcp_client, + # output_descriptors=[ListProperty(name="generate_listOutput")], # this will be automatically inferred + ) + + +You can then use those tools in a :ref:`Flow ` to natively support the manipulation of complex data types with MCP tools. + +You can also use Pydantic models to change the tool output names. Note that in this advanced use, +you must wrap the outputs in a `result` field as expected by MCP when using non-dict types. +This also enables the use of multi-output in tools by using tuples. + + +.. code-block:: python + + from typing import Annotated + from pydantic import BaseModel, RootModel, Field + + class GenerateTupleOut(BaseModel, title="tool_output"): + result: tuple[ + Annotated[str, Field(title="str_output")], + Annotated[bool, Field(title="bool_output")] + ] + # /!\ this needs to be named `result` + + class GenerateListOut(BaseModel, title="tool_output"): + result: list[str] # /!\ this needs to be named `result` + + class GenerateDictOut(RootModel[dict[str, str]], title="tool_output"): + pass + + server = FastMCP( + name="Example MCP Server", + instructions="A MCP Server.", + host=host, + port=port, + ) + + @server.tool(description="Tool that generates a dictionary", structured_output=True) + def generate_dict() -> GenerateDictOut: + return GenerateDictOut({"key": "value"}) + + @server.tool(description="Tool that generates a list", structured_output=True) + def generate_list() -> GenerateListOut: + return GenerateListOut(result=["value1", "value2"]) + + @server.tool(description="Tool that returns multiple outputs", structured_output=True) + def generate_tuple(inputs: list[str]) -> GenerateTupleOut: + value = "; ".join(inputs) + return GenerateTupleOut(result=("value", True)) + + +You can then match the output descriptors on the WayFlow side. + +.. code-block:: python + + generate_dict_tool = MCPTool( + name="generate_dict", + description="Tool that generates a dictionary", + client_transport=mcp_client, + output_descriptors=[DictProperty(name="tool_output")], + ) + + generate_list_tool = MCPTool( + name="generate_list", + description="Tool that generates a list", + client_transport=mcp_client, + output_descriptors=[ListProperty(name="tool_output")], + ) + + generate_tuple_tool = MCPTool( + name="generate_tuple", + description="Tool that returns multiple outputs", + client_transport=mcp_client, + input_descriptors=[ListProperty(name="inputs")], + output_descriptors=[StringProperty(name="str_output"), BooleanProperty(name="bool_output")], + ) + +When specified, the input/output descriptors of the MCP tool will be validated against the schema fetched from the MCP server. + + +.. note:: + + MCPToolBox is not compatible with complex output types. + Tools from MCPToolBox will always return string values. + + +Exporting/Loading with Agent Spec +================================= + +You can export the assistant from this tutorial to Agent Spec: + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Export_config_to_Agent_Spec + :end-before: # .. end-##_Export_config_to_Agent_Spec + + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: Agent Configuration + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_mcp_agent.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_mcp_agent.yaml + :language: yaml + + .. tab:: Flow Configuration + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_mcp_flow.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_mcp_flow.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. +Because this example uses an unauthenticated local MCP server, the load step is wrapped +in ``authless_mcp_enabled()`` too. + + +.. literalinclude:: ../code_examples/howto_mcp.py + :language: python + :start-after: # .. start-##_Load_Agent_Spec_config + :end-before: # .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginMCPToolBox`` + - ``ExtendedToolNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next Steps +========== + +Having learned how to integrate MCP servers in WayFlow, you may now proceed to: + +- :doc:`How to Enable Tool Output Streaming ` +- :doc:`How to Add User Confirmation to Tool Call Requests ` +- :doc:`How to Create a ServerTool from a Flow ` + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_mcp.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_multiagent.rst.txt b/26.1.2/_sources/core/howtoguides/howto_multiagent.rst.txt new file mode 100644 index 000000000..71330fa3b --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_multiagent.rst.txt @@ -0,0 +1,295 @@ +============================================== +How to Build a Hierarchical Multi-Agent System +============================================== + + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Agents <../tutorials/basic_agent>`. + + +With the advent of increasingly powerful Large Language Models (LLMs), multi-agent systems are becoming more relevant +and are expected to be particularly valuable in scenarios requiring high-levels of autonomy and/or processing +of diverse sources of information. + +There are various types of multi-agent systems, each serving different purposes and applications. +Some notable examples include hierarchical structures, agent swarms, and mixtures of agents. + +This guide demonstrates an example of a hierarchical multi-agent system (also known as manager-worker pattern) and will show you how to: + +- Build expert agents equipped with tools; +- Test the expert agents individually; +- Build and test a hierarchical multi-agent assistant. + +.. image:: ../_static/howto/howto_multiagent.svg + :align: center + :scale: 70% + :alt: Example of a multi-agent system + +**Diagram:** Multi-agent system shown in this how-to guide, comprising a manager agent +(customer service agent) and two expert agents equipped with tools (refund specialist +and satisfaction surveyor). + +.. seealso:: + + To access short code snippets demonstrating how to use other agentic patterns in WayFlow, + refer to the :doc:`Reference Sheet <../misc/reference_sheet>`. + + +To follow this guide, you need an LLM. WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Building and testing expert Agents +================================== + +In this guide you will use the following helper function to print +messages: + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-helpermethod: + :end-before: .. end-helpermethod + +API Reference: :ref:`MessageType ` + + +Refund specialist agent +----------------------- + +Tools +^^^^^ + +The refund specialist agent is equipped with two tools. + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialisttools: + :end-before: .. end-specialisttools + +API Reference: :ref:`tool ` + +The first tool is used to check whether a given order is eligible for a refund, +while the second is used to process the specific refund. + + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialistprompt: + :end-before: .. end-specialistprompt + +.. important:: + + The quality of the system prompt is paramount to ensuring proper behaviour of the multi-agent + system, because slight deviations in the behaviour can lead to cascading + unintended effects as the number of agents scales up. + + +Building the Agent +^^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialistagent: + :end-before: .. end-specialistagent + +API Reference: :ref:`Agent ` + + +Testing the Agent +^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialisttest: + :end-before: .. end-specialisttest + +Test the agents individually to ensure they perform as expected. + + +Statisfaction surveyor agent +---------------------------- + +Tools +^^^^^ + +The statisfaction surveyor agent is equipped with one tool. + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyortools: + :end-before: .. end-surveyortools + + +The ``record_survey_response`` tool is simulating the recording of +user feedback data. + + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyorprompt: + :end-before: .. end-surveyorprompt + + +Building the Agent +^^^^^^^^^^^^^^^^^^ + + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyoragent: + :end-before: .. end-surveyoragent + + +Testing the Agent +^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyortest: + :end-before: .. end-surveyortest + + +Again, the expert agent behaves as intended. + + +Building and testing the multi-agent assistant +============================================== + + +Manager Agent +------------- + +In WayFlow, a hierarchical multi-agent system can simply be created as +an agent equipped with expert agents (the ones defined earlier). + +This agent is also known as **manager** or **router** agent. + +System prompt +^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managerprompt: + :end-before: .. end-managerprompt + +Building the manager Agent +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-manageragent: + :end-before: .. end-manageragent + + +Testing the multi-agent system +------------------------------ + +The power of mult-agent systems is their high adaptiveness. +In the following example, it is demonstrated how the manager can decide +not to call the expert agents for simple user queries. + + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managertest_noexpert: + :end-before: .. end-managertest_noexpert + + +However, the manager is explicitly prompted to delegate to the specialized agents for more complex tasks. +This is demonstrated in the following example. + + +.. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managertest_withexpert: + :end-before: .. end-managertest_withexpert + + + +Recap +===== + +In this guide, you learned how to build a multi-agent system +consisting of a manager agent and two expert sub-agents. + +.. collapse:: Below is the complete code from this guide. + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-helpermethod: + :end-before: .. end-helpermethod + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialisttools: + :end-before: .. end-specialisttools + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialistprompt: + :end-before: .. end-specialistprompt + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialistagent: + :end-before: .. end-specialistagent + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-specialisttest: + :end-before: .. end-specialisttest + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyortools: + :end-before: .. end-surveyortools + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyorprompt: + :end-before: .. end-surveyorprompt + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyoragent: + :end-before: .. end-surveyoragent + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-surveyortest: + :end-before: .. end-surveyortest + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managerprompt: + :end-before: .. end-managerprompt + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-manageragent: + :end-before: .. end-manageragent + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managertest_noexpert: + :end-before: .. end-managertest_noexpert + + .. literalinclude:: ../code_examples/howto_multiagent.py + :language: python + :start-after: .. start-managertest_withexpert: + :end-before: .. end-managertest_withexpert + + +Next steps +========== + +Having learned how to build a multi-agent system in WayFlow, you may now proceed to :doc:`How to Use Agents in Flows `. diff --git a/26.1.2/_sources/core/howtoguides/howto_multiple_output_tool.rst.txt b/26.1.2/_sources/core/howtoguides/howto_multiple_output_tool.rst.txt new file mode 100644 index 000000000..83b357bfd --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_multiple_output_tool.rst.txt @@ -0,0 +1,118 @@ +========================================= +How to Create Tools with Multiple Outputs +========================================= + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Tools <../api/tools>` + - :doc:`Building Assistants with Tools ` + +:doc:`WayFlow Tools <../api/tools>` are a powerful way to equip assistants with new, controllable capabilities. +A key feature of tools is their ability to return multiple outputs, which can then be used across different steps in a `:doc:Flow <../tutorials/basic_flow>`. +Understanding how to map these outputs to variables within a Flow is useful for creating flexible and modular assistants. + +In this tutorial, you will: + +- Recap how tools output their variables +- Learn how to output multiple variables from a tool with the tool annotation +- Learn how to output multiple variables from a tool with `ServerTool` + +Imports +======= + +To get started, import the following elements: + +.. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-imports_multioutput_tool + :end-before: .. end-imports_multioutput_tool + +Basic tool implementation +========================= + +When using a tool decorator, the tool step returns a **single output** by default. +If the tool's return type is a dictionary (Dict), the step returns a dictionary object. +This output can be accessed using ``ToolExecutionStep.TOOL_OUTPUT``, which is its default name. +While this is useful in many cases, it does not allow the flow to access each variable individually without additional preprocessing. + +.. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-basic_tool_decorator + :end-before: .. end-basic_tool_decorator + +As can be seen in the next two sections, it is possible to use either the `@tool` annotation or `ServerTool` to be able to output several outputs out of a tool. + +Multiple outputs with the tool annotation +========================================= + +To enable a tool to return multiple outputs that can be used directly in subsequent steps, all expected outputs must be defined using ``output_descriptors``. +This argument can be passed to the `@tool` annotation. +The flow will then unpack the dictionary returned by the tool and assigns each value to a separate variable. + +.. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-multi_output_decorator + :end-before: .. end-multi_output_decorator + +.. warning:: + If ``output_descriptors`` receives a single variable (i.g., ``[StringProperty(...)]``), the dictionary will not be unpacked. + Instead, the entire dictionary will be treated as a single output, which may result in a type error. + +In this case, the unpacked output becomes ``{'bool_output': False, 'string_output': 'Hello World!', 'integer_output': 2147483647}``, and we can use each of the variables independently inside the Flow. + + +Multiple outputs with `ServerTool` +================================== + +To enable a tool to return multiple outputs that can be used directly in subsequent steps, all expected outputs must be defined using ``output_descriptors``. +The model will then unpack the dictionary returned by the tool and assigns each value to a separate variable. + +.. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-multi_output_server + :end-before: .. end-multi_output_server + +.. warning:: + If ``output_descriptors`` receives a single variable (i.g., ``[StringProperty(...)]``), the dictionary will not be unpacked. + Instead, the entire dictionary will be treated as a single output, which may result in a type error. + +In this case, the unpacked output becomes ``{'bool_output': False, 'string_output': 'Hello World!', 'integer_output': 2147483647}``, and we can use each of the variables independently inside the Flow. + +Recap +===== + +In this guide, you have learned how to extract individual variables returned by a `ToolExecutionStep` so they can be used independently within a ``Flow``. + +.. collapse:: Below is the complete code referenced in this guide. + + .. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-imports_multioutput_tool + :end-before: .. end-imports_multioutput_tool + + .. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-multi_output_decorator + :end-before: .. end-multi_output_decorator + + .. literalinclude:: ../code_examples/howto_multiple_output_tool.py + :language: python + :linenos: + :start-after: .. start-multi_output_server + :end-before: .. end-multi_output_server + +Next steps +========== + +Now that you have learned how to produce multiple outputs in a ``Tool`` and use them in a ``Flow``, you may proceed to: + +- Using a :ref:`BranchingStep ` to decide how to handle those variables. +- :doc:`How to Do Map and Reduce Operations in Flows `. diff --git a/26.1.2/_sources/core/howtoguides/howto_ociagent.rst.txt b/26.1.2/_sources/core/howtoguides/howto_ociagent.rst.txt new file mode 100644 index 000000000..3fb5e34f2 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_ociagent.rst.txt @@ -0,0 +1,109 @@ +.. _top-howtoociagent: + +=================================== +How to Use OCI Generative AI Agents +=================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_ociagent.py + :link-alt: OCI Agent how-to script + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLM configuration <../howtoguides/llm_from_different_providers>` + - :doc:`Using agents ` + +`OCI GenAI Agents `_ is a service to create agents in the OCI console. +These agents are defined remotely, including their tools, prompts, and optional documents for retrieval-augmented generation (RAG), and can be used for inference. + +In this guide, you will learn how to connect an OCI agent using the :ref:`OciAgent ` class from the ``wayflowcore`` package. + + +Basic usage +=========== + +To get started, first create your OCI Agent in the OCI Console. +Consult the OCI documentation for detailed steps: https://docs.oracle.com/en-us/iaas/Content/generative-ai-agents/home.htm. + +Next, create an ``OciClientConfig`` object to configure the connection to the OCI service. +See the :doc:`OCI LLM configuration ` for detailed instructions how to configure this object. + +You will also need the ``agent_endpoint_id`` from the OCI Console. +This ID points to the agent you want to connect to, while the client configuration is about connecting to the entire service. + +Once these are in place, you can create your agent in a few lines: + +.. literalinclude:: ../code_examples/howto_ociagent.py + :language: python + :start-after: .. start-##_Creating_the_agent + :end-before: .. end-##_Creating_the_agent + +Then, use the agent as shown below: + +.. literalinclude:: ../code_examples/howto_ociagent.py + :language: python + :start-after: .. start-##_Running_the_agent + :end-before: .. end-##_Running_the_agent + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_ociagent.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_ociagent.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_ociagent.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_ociagent.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Next steps +========== + +Now that you have learned how to use OCI agents in WayFlow, you may proceed to :doc:`How to Use Agents in Flows `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_ociagent.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_parallelflowexecution.rst.txt b/26.1.2/_sources/core/howtoguides/howto_parallelflowexecution.rst.txt new file mode 100644 index 000000000..2ed892dcd --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_parallelflowexecution.rst.txt @@ -0,0 +1,177 @@ +.. _top-howtoparallelflowexecution: + +===================================== +How to Run Multiple Flows in Parallel +===================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_parallelflowexecution.py + :link-alt: Parallel flow execution how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +Parallelism is a fundamental concept in computing that enables tasks to be processed concurrently, +significantly enhancing system efficiency, scalability, and overall performance. + +WayFlow supports the execution of multiple Flows in parallel, using the :ref:`ParallelFlowExecutionStep `. +This guide will show you how to: + +- use :ref:`ParallelFlowExecutionStep ` to run several tasks in parallel +- use :ref:`PromptExecutionStep ` to summarize the outcome of the parallel tasks + +To follow this guide, you need an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Basic implementation +==================== + +In this guide, we will create a ``Flow`` that generates a marketing message for a user. +Taking the username that identifies the user as input, we will take advantage of the ``ParallelFlowExecutionStep`` +to concurrently retrieve information about the user and the context, so that we can finally generate a +personalized marketing welcome message. + +We first define the following tools that retrieve the desired information: + +* One tool that retrieves the current time; +* One tool that retrieves the user information, like name and date of birth; +* One tool that gathers the user's purchase history; +* One tool that looks for the current list of items on sale, which could be recommended to the user. + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Define_the_tools + :end-before: .. end-##_Define_the_tools + +These tools simply gather information, therefore they can be easily parallelized. +We create the flows that wrap the tools we just created, and we collect them all in a ``ParallelFlowExecutionStep`` +for parallel execution. + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Create_the_flows_to_be_run_in_parallel + :end-before: .. end-##_Create_the_flows_to_be_run_in_parallel + +The ``ParallelFlowExecutionStep`` will expose all the outputs that the different inner flows generate. +We use this information to ask an LLM to generate a personalized welcome message for the user, which should also +have a marketing purpose. + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Generate_the_marketing_message + :end-before: .. end-##_Generate_the_marketing_message + +Now that we have all the steps that compose our flow, we just put everything together to create it, and we +execute it to generate our personalized message. + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Create_and_test_the_final_flow + :end-before: .. end-##_Create_and_test_the_final_flow + + +Notes about parallelization +=========================== + +Not all sub-flows can be executed in parallel. +The table below summarizes the limitations of parallel execution for the :ref:`ParallelFlowExecutionStep `: + + .. list-table:: + :widths: 30 50 50 45 + :header-rows: 1 + + * - Support + - Type of flow + - Examples + - Remarks + * - **FULLY SUPPORTED** + - **Flows that do not yield and do not have any side-effect on the conversation** (no variable read/write, posting to the conversation, and so on) + - Embarrassingly parallel flows (simple independent operation), such as a ``PromptExecutionStep``, ``ApiCallStep`` to post or get, and so on + - N/A + * - **SUPPORTED WITH SIDE EFFECTS** + - **Flows that do not yield but have some side-effect on the conversation** (variable read/write, posting to the conversation, and so on) + - Flows with ``OutputMessageStep``, ``VariableStep``, ``VariableReadStep``, ``VariableWriteStep``, and so on + - No guarantee in the order of operations (such as posting to the conversation), only the outputs are guaranteed in order. + * - **NON SUPPORTED** + - **Flows that yield**. WayFlow does not support this, otherwise a user might be confused in what branch they are currently when prompted to answer. + - Flows with ``InputMessageStep``, ``AgentExecutionStep`` that can ask questions, ``ClientTool``, and so on + - It will raise an exception at instantiation time if a sub-flow can yield and step set to parallel + +.. note:: + The Global Interpreter Lock (GIL) in Python is not a problem for parallel remote requests + because I/O-bound operations, such as network requests, release the GIL during their execution, + allowing other threads to run concurrently while the I/O operation is in progress. + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_parallelflowexecution.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_parallelflowexecution.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_parallelflowexecution.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``ExtendedParallelFlowNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Having learned how to perform generic parallel operations in WayFlow, you may now proceed to +:doc:`How to Do Map and Reduce Operations in Flows `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code +for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_parallelflowexecution.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_plugins.rst.txt b/26.1.2/_sources/core/howtoguides/howto_plugins.rst.txt new file mode 100644 index 000000000..173e07dcc --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_plugins.rst.txt @@ -0,0 +1,159 @@ +.. _top-howtoplugins: + +==================================== +How to Create New WayFlow Components +==================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_plugins.py + :link-alt: WayFlow plugins how-to script + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLM configuration <../howtoguides/llm_from_different_providers>` + - :doc:`Using agents ` + +WayFlow Plugins are the expected mean that users can use to introduce new concepts and components, or extensions to existing ones, +such that they can be integrated seamlessly into the serialization, deserialization, and Agent Spec conversion processes +of WayFlow. + +In this guide, you will learn how to build a WayFlow plugin to introduce a new custom component in WayFlow, and make sure +that it can be serialized, deserialized, and converted to Agent Spec. + +You are going to build a specialized :ref:`ServerTool ` that reads the content of a file given its path. +Then we are going to use this tool as part of an :ref:`Agent `, build the plugins, and show how to use them +for WayFlow's serialization and Agent Spec conversion. + +Basic usage +=========== + +As first step, we build our new tool extension. We call it ``ReadFileTool``, and we give it an attribute +to specify which file extensions are allowed to be read. The tool implementation in this example is mocked, +and it just returns the content of two predefined filepaths. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Create_the_new_tool_to_read_a_file + :end-before: .. end-##_Create_the_new_tool_to_read_a_file + +In this example we extended ``Tool``, but a new component that does not inherit from existing concepts can be created, +and it can be connected to the serialization mechanism by simply extending the ``SerializableObject`` interface. +Note that there are also extensions of the ``SerializableObject`` class, like the ``SerializableDataclass``, +which offers basic serialization implementation for ``dataclass`` annotated classes. + +We can now build our agent: we select the LLM we want to use to orchestrate it, write custom instructions to +inform it about which files are available, and give it instructions to read them if needed. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Create_the_agent + :end-before: .. end-##_Create_the_agent + +As already mentioned at the beginning of this guide, WayFlow plugins take also care of the Agent Spec conversion. +Therefore, in order to create complete WayFlow plugins, we have to create the new tool also in Agent Spec. +To do that, we extend the ``ServerTool`` implementation from ``pyagentspec`` and we add the ``allowed_extensions`` attribute. +Then we create the Agent Spec plugins that will take care of the serialization/deserialization to/from Agent Spec. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Create_Agent_Spec_components_and_plugins + :end-before: .. end-##_Create_Agent_Spec_components_and_plugins + +Now that the Agent Spec plugins are ready, we can use them in our WayFlow plugins for the ReadFileTool. +We build the :ref:`WayflowSerializationPlugin ` and the :ref:`WayflowDeserializationPlugin ` +by specifying a plugin name, a version, the Agent Spec plugins (if any) that are required to serialize and deserialize +the new Tool, and implement the conversion logic for it. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Create_Wayflow_plugins_for_serialization_and_Agent_Spec_conversion + :end-before: .. end-##_Create_Wayflow_plugins_for_serialization_and_Agent_Spec_conversion + +Now that we have built the plugins, we can use them in serialization and deserialization. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Serialize_and_deserialize_the_agent + :end-before: .. end-##_Serialize_and_deserialize_the_agent + +Finally, we can try to run our agent and ask it some information contained in a file. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Agent_Execution + :end-before: .. end-##_Agent_Execution + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_plugins.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_plugins.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_plugins.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``AgentPlugin`` + - ``MessageTransformPlugin`` + - ``PromptTemplatePlugin`` + - ``OutputParserPlugin`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Now that you have learned how to build WayFlow plugins, you may proceed to +:doc:`Load and Execute an Agent Spec Configuration `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_plugins.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_promptexecutionstep.rst.txt b/26.1.2/_sources/core/howtoguides/howto_promptexecutionstep.rst.txt new file mode 100644 index 000000000..184dc3d50 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_promptexecutionstep.rst.txt @@ -0,0 +1,323 @@ +.. _top-howtopromptexecutionstep: + +============================================ +How to Do Structured LLM Generation in Flows +============================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_promptexecutionstep.py + :link-alt: Prompt execution step how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +WayFlow enables you to leverage LLMs to generate text and structured outputs. +This guide will show you how to: + +- use the :ref:`PromptExecutionStep ` to generate text using an LLM +- use the :ref:`PromptExecutionStep ` to generate structured outputs +- use the :ref:`AgentExecutionStep ` to generate structured outputs using an agent + + +Basic implementation +==================== + +In this how-to guide, you will learn how to perform structured LLM generation with Flows. + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Assuming you want to summarize this article: + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Define_the_article + :end-before: .. end-##_Define_the_article + +WayFlow offers the :ref:`PromptExecutionStep ` for this type of query. +Use the code below to generate a 10-word summary: + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Create_the_flow_using_the_prompt_execution_step + :end-before: .. end-##_Create_the_flow_using_the_prompt_execution_step + +.. note:: + + In the prompt, ``article`` is a Jinja2 syntax to specify a placeholder for a variable, which will appear as an input for the step. + If you use ``{{var_name}}``, the variable named ``var_name`` will be of type ``StringProperty``. + If you specify anything else Jinja2 compatible (for loops, filters, and so on), it will be of type ``AnyProperty``. + +Now execute the flow: + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Run_the_flow_to_get_the_summary + :end-before: .. end-##_Run_the_flow_to_get_the_summary + +As expected, your flow has generated the article summary! + +Structured generation with Flows +================================ + +In many cases, generating raw text within a flow is not very useful, as it is difficult to leverage in later steps. +Instead, you might want to generate attributes that follow a particular schema. +The ``PromptExecutionStep`` class enables this through the `output_descriptors` parameter. + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Use_structured_generation_to_extract_formatted_information + :end-before: .. end-##_Use_structured_generation_to_extract_formatted_information + + +Complex JSON objects +==================== + +Sometimes, you might need to generate an object that follows a specific JSON Schema. +You can do that by using an output descriptor of type ``ObjectProperty``, or directly converting your JSON Schema into a descriptor: + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Use_structured_generation_with_JSON_schema + :end-before: .. end-##_Use_structured_generation_with_JSON_schema + + +Structured generation with Agents +================================= + +In certain scenarios, you might need an agent to generate well-formatted outputs within your flow. +You can instruct the agent to generate them, and use it in the ``AgentExecutionStep`` class to perform structured generation. + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Use_structured_generation_with_Agents_in_flows + :end-before: .. end-##_Use_structured_generation_with_Agents_in_flows + +How to write secure prompts with Jinja templating +================================================= + +.. _securejinjatemplating: + +`Jinja2 `_ is a fast and flexible templating engine for Python, +enabling dynamic generation of text-based formats by combining templates with data. + +However, enabling all Jinja templating capabilities poses some security challenges. +For this reason, WayFlow relies on a stricter implementation of the Jinja2's SandboxedEnvironment for higher security. +Every callable is considered unsafe, and every attribute and item access is prevented, except for: + +* The attributes ``index0``, ``index``, ``first``, ``last``, ``length`` of the ``jinja2.runtime.LoopContext``; +* The entries of a Python dictionary (only native type is accepted); +* The items of a Python list (only native type is accepted). + +You should never write a template that includes a function call, or access to any internal attribute or element of +an arbitrary variable: that is considered unsafe, and it will raise a ``SecurityException``. + +Moreover, WayFlow performs additional checks on the inputs provided for rendering. +In particular, only elements and sub-elements that are of basic Python types +(``str``, ``int``, ``float``, ``bool``, ``list``, ``dict``, ``tuple``, ``set``, ``NoneType``) are accepted. +In any other case, a ``SecurityException`` is raised. + +What you can write +------------------ + +Here's a set of common patterns that are accepted by WayFlow's restricted Jinja templating. + +Templates that access variables of base Python types: + + .. code-block:: python + + my_var: str = "simple string" + template = "{{ my_var }}" + # Expected outcome: "simple string" + +Templates that access elements of a list of base Python types: + + .. code-block:: python + + my_var: list[str] = ["simple string"] + template = "{{ my_var[0] }}" + # Expected outcome: "simple string" + +Templates that access dictionary entries of base Python types: + + .. code-block:: python + + my_var: dict[str, str] = {"k1": "simple string"} + template = "{{ my_var['k1'] }}" + # Expected outcome: "simple string" + + my_var: dict[str, str] = {"k1": "simple string"} + template = "{{ my_var.k1 }}" + # Expected outcome: "simple string" + +Builtin functions of Jinja, like ``length`` or ``format``: + + .. code-block:: python + + my_var: list[str] = ["simple string"] + template = "{{ my_var | length }}" + # Expected outcome: "1" + +Simple expressions: + + .. code-block:: python + + template = "{{ 7*7 }}" + # Expected outcome: "49" + +``For`` loops, optionally accessing the ``LoopContext``: + + .. code-block:: python + + my_var: list[int] = [1, 2, 3] + template = "{% for e in my_var %}{{e}}{{ ', ' if not loop.last }}{% endfor %}" + # Expected outcome: "1, 2, 3" + +``If`` conditions: + + .. code-block:: python + + my_var: int = 4 + template = "{% if my_var % 2 == 0 %}even{% else %}odd{% endif %}" + # Expected outcome: "even" + +Our general recommendation is to avoid complex logic in templates, and to pre-process the data you want to render instead. +For example, in case of complex objects, in order to comply with restrictions above, you should conveniently +transform them recursively into a dictionary of entries of basic Python types (see list of accepted types above). + +What you cannot write +--------------------- + +Here's a set of common patterns that are **NOT** accepted by WayFlow's restricted Jinja templating. + +Templates that access arbitrary objects: + + .. code-block:: python + + my_var: MyComplexObject = MyComplexObject() + template = "{{ my_var }}" + # Expected outcome: SecurityException + +Templates that access attributes of arbitrary objects: + + .. code-block:: python + + my_var: MyComplexObject = MyComplexObject(attribute="my string") + template = "{{ my_var.attribute }}" + # Expected outcome: SecurityException + +Templates that access internals of any type and object: + + .. code-block:: python + + my_var: dict = {"k1": "my string"} + template = "{{ my_var.__init__ }}" + # Expected outcome: SecurityException + +Templates that access non-existing keys of a dictionary: + + .. code-block:: python + + my_var: dict = {"k1": "my string"} + template = "{{ my_var['non-existing-key'] }}" + # Expected outcome: SecurityException + +Templates that access keys of a dictionary of type different from ``int`` or ``str``: + + .. code-block:: python + + my_var: dict = {("complex", "key"): "my string"} + template = "{{ my_var[('complex', 'key')] }}" + # Expected outcome: SecurityException + +Templates that access callables: + + .. code-block:: python + + my_var: Callable = lambda x: f"my value {x}" + template = "{{ my_var(2) }}" + # Expected outcome: SecurityException + + my_var: list = [1, 2, 3] + template = "{{ len(my_var) }}" + # Expected outcome: SecurityException + + my_var: MyComplexObject = MyComplexObject() + template = "{{ my_var.to_string() }}" + # Expected outcome: SecurityException + + +For more information, please check our :doc:`Security considerations page <../security>`. + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_promptexecutionstep.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_promptexecutionstep.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_promptexecutionstep.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +Recap +===== + +In this guide, you learned how to incorporate LLMs into flows to: + +- generate raw text +- produce structured output +- perform structured generation using the agent and :ref:`AgentExecutionStep ` + + +Next steps +========== + +Having learned how to perform structured generation in WayFlow, you may now proceed to: + +- :doc:`Config Generation ` to change LLM generation parameters. +- :doc:`Catching Exceptions ` to ensure robustness of the generated outputs. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_promptexecutionstep.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_prompttemplate.rst.txt b/26.1.2/_sources/core/howtoguides/howto_prompttemplate.rst.txt new file mode 100644 index 000000000..6fe76e188 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_prompttemplate.rst.txt @@ -0,0 +1,297 @@ +.. _top-howtoprompttemplates: + +======================================== +How to Use Advanced Prompting Techniques +======================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_prompttemplate.py + :link-alt: Prompt templates how-to script + + Python script/notebook for this guide. + + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLMs <../api/llmmodels>` + - :doc:`Agent <../api/agent>` + +:doc:`PromptTemplate <../api/prompttemplate>` is a powerful way to configure how to prompt various LLMs. +Although WayFlow has very good defaults for most tasks and LLM providers, some specific use cases or providers require adapting prompt formatting to improve efficiency (e.g., token cost) or performance. +To address this, WayFlow introduces the concept of ``PromptTemplate``, which enables you to define prompts in a very generic way, applicable across different scenarios. + +This guide will show you how to: + +- Recap how to use variables in prompts +- Learn how to configure a custom ``PromptTemplate`` +- Learn how to configure a prompt template for an ``Agent`` +- Learn best practices for writing prompts + +Basics of prompting +=================== + +WayFlow leverages LLMs to perform chat completion, meaning that given a list of messages (which can be ``system`` (for instructions to the LLM), +``agent`` (a message generated by the LLM), and ``user`` (a message from the user)), it returns a new message. + +To perform agentic tasks, LLMs are equipped with ``tools``, which are defined functions with names and arguments that the LLM may decide to call. +Prompting the LLM to generate such calls can be done using ``native_tool_calling``, where the LLM provider handles how tools are presented to the model and how tool calls are parsed, +or through custom tool calling, where the presentation of tools to the model is controlled externally and the raw text output from the model must be parsed manually. + +In many use cases, generating raw text is not enough because it is difficult to use in further pipeline steps. +LLMs can produce specific structured output, often through ``structured_generation``. +Most LLM providers support this feature natively (the provider takes care of formatting the expected response and parsing the LLM output), +but you can also do it in a custom way, using a specific prompt combined with custom parsing. + +The ``PromptTemplate`` abstraction is designed so that configuring all these parameters is as simple as possible, while guaranteeing maximum customization. +Using ``PromptTemplate`` usually involves 3 steps: + +- Create the basic template. This is done using either the constructor (``PromptTemplate()``) or the helper function from a string (``PromptTemplate.from_string()``). +- Equip the template with ``partial values``, ``tools``, or ``response_format``. This can be done dynamically, and is useful when these elements are unknown at template creation but can be reused across multiple generations. +- Render the template: use ``template.format()`` to fill all variables and create a ``Prompt`` object, which can then be passed to an LLM (e.g., ``llm.generate(prompt)``). This object contains all the information needed for the LLM generation. + +Configure the LLM +================= + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Basic text prompt with Regex parsing +==================================== + +One possible use case is prompting an LLM and extracting some information from the raw text output. +This is especially relevant in a prompt template, since the parsing method highly depends on how the LLM is prompted in the first place. + +The example below shows how to write a simple ``Chain-of-Thoughts`` prompt. +The ``RegexOutputParser`` is used to configure how output is extracted from the raw text. + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-##_Basic_text_prompt_with_Regex_parsing + :end-before: .. end-##_Basic_text_prompt_with_Regex_parsing + +API reference: :ref:`PromptTemplate `, :ref:`RegexOutputParser `, :ref:`RegexPattern `, :ref:`JsonOutputParser `. + + +Prompt with chat history +======================== + +In many cases—especially when working with agents—it is necessary to format a list of messages within a prompt. +There are two ways to achieve this. +Assume a list of messages is available: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-##_Prompt_with_chat_history + :end-before: .. end-##_Prompt_with_chat_history + +The first way is to integrate the history of messages directly into the prompt's list of messages. +This helps the model by presenting the entire conversation as a single context. +For example, to format the conversation as messages within a template, you can use the ``CHAT_HISTORY_PLACEHOLDER``: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_As_inlined_messages + :end-before: .. end-###_As_inlined_messages + +You can also format the chat history directly in a message, by using the ``CHAT_HISTORY_PLACEHOLDER_NAME`` placeholder (whose underlying value is the string ``"__CHAT_HISTORY__"`` as shown in the code snippet below): + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_In_the_system_prompt + :end-before: .. end-###_In_the_system_prompt + +If you need to filter some part of the chat history before rendering it in the template, you may use some pre-rendering message transforms: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_message_transform + :end-before: .. end-###_With_message_transform + +API reference: :ref:`MessageTransform `. + +This will ensure only the last chat history message is formatted in the prompt. + +Configure how to use tools in templates +======================================= + +Assume a tool is available and should be used for LLM generation. + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-##_Configure_how_to_use_tools_in_templates + :end-before: .. end-##_Configure_how_to_use_tools_in_templates + +If the LLM provider supports ``native_tool_calling``, integrating it into templates is straightforward: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_native_tool_calling + :end-before: .. end-###_With_native_tool_calling + +In this case, the template uses native tool calling, so the prompt includes separate ``tools`` that are passed to the LLM endpoint. +The provider directly parses the output and returns a ``ToolRequest``. + +Sometimes, the provider's native tool calling might not be available or may not work as needed (for example, when performing Chain-of-Thought reasoning with Llama native models, which do not support this feature). +In such cases, tool calling can be configured directly within the prompt template. + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_custom_tool_calling + :end-before: .. end-###_With_custom_tool_calling + +For custom tool calling, the important parameters to configure are: + +- ``native_tool_calling`` indicates that the tools will be formatted in the prompt. This requires the ``__TOOLS__`` placeholder in one of the messages. +- ``post_rendering_transforms``: transformations applied to messages just before passing them to the LLM. These allow transformations specific to a tool-calling technique or an LLM provider's requirements. The following post-transforms are used here: + + - ``_ReactMergeToolRequestAndCallsTransform`` formats tool calls and tool requests into standard user or agent messages. + - ``CoalesceSystemMessagesTransform`` allows merging consecutive system messages into a single system message, as required by some LLM providers. + - ``RemoveEmptyNonUserMessageTransform`` removes empty agent or system messages, since some LLMs do not support them. +- ``output_parser``: specifies how the raw LLM text output is parsed into a tool request. Several examples can be found in the ``wayflowcore.templates`` subpackage. +- ``generation_config``: for this ReAct-style template, specific generation parameters are added to help reduce hallucinations. + + +Configure how to use structured generation in templates +======================================================= + +Structured generation can be handled using either ``native_tool_calling`` (if the LLM provider supports it) or custom methods. +Assume the goal is to generate the following output: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-##_Configure_how_to_use_structured_generation_in_templates + :end-before: .. end-##_Configure_how_to_use_structured_generation_in_templates + +API reference: :ref:`ObjectProperty `, :ref:`StringProperty `. + +With native structured generation +--------------------------------- + +Some providers allow passing the desired output format separately and ensure the generated output follows the given format. +If the LLM provider supports it, then using a template is straightforward: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_native_structured_generation + :end-before: .. end-###_With_native_structured_generation + +With custom structured generation +--------------------------------- + +If the LLM provider does not support native structured generation, it is possible to prompt the LLM to produce output in the desired format manually: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_custom_structured_generation + :end-before: .. end-###_With_custom_structured_generation + +The main parameters to configure are: + +- ``native_structured_generation=False``: prevents the ``response_format`` from being passed separately. If a ``__RESPONSE_FORMAT__`` placeholder is used, the ``response_format`` will be inserted there. +- ``output_parser``: to correctly parse the raw LLM output into the expected object. In this example, a simple output parser is used that can repair malformed JSON output. + +You can also use custom output parser to ensure the expected format. +In such cases, it is beneficial to include the ``response_format`` in the template to clearly indicate the structured output format being generated. +For example, parsing the content of the chain-of-thoughts can be done as follows: + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :linenos: + :start-after: .. start-###_With_additional_output_parser + :end-before: .. end-###_With_additional_output_parser + +.. tip:: + Wayflow also provides a helper function to set up a simple JSON structured generation prompt and + the associated output parser from an existing prompt template leveraging native structured generation. + Check out the :ref:`helper method's API Reference `. + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_prompttemplate.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_prompttemplate.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_prompttemplate.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginPromptTemplate`` + - ``PluginRegexOutputParser`` + - ``ExtendedAgent`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +.. note:: + By default, all WayFlow serialization/deserialization plugins are used when loading or exporting + from/to Agent Spec. Passing a list of plugins overrides the default ones. + + +Next steps +========== + +Having learned how to use the ``PromptTemplate``, you may now proceed to using it in :ref:`PromptExecutionStep ` or :ref:`Agents `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_prompttemplate.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_rag.rst.txt b/26.1.2/_sources/core/howtoguides/howto_rag.rst.txt new file mode 100644 index 000000000..2d69eccac --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_rag.rst.txt @@ -0,0 +1,592 @@ +.. _top-rag: + +============================================ +How to Build RAG-Powered Assistants +============================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_rag.py + :link-alt: RAG how-to script + + Python script/notebook for this guide. + + +.. admonition:: Prerequisites + + This guide assumes familiarity with + + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Datastores ` + + +Retrieval-Augmented Generation (RAG) is a powerful technique that enhances AI assistants by connecting them to external knowledge sources. +Instead of relying solely on their training data, RAG-enabled assistants can search through your specific documents, databases, or knowledge bases to provide accurate, up-to-date, and contextually relevant responses. + +In this tutorial, you will: + +- **Configure vector search** to enable semantic similarity matching in your data. +- **Create a searchable datastore** with embeddings for efficient retrieval. +- **Create a RAG-powered Agent** that autonomously searches for information to fulfill user requests. +- **Build a RAG-powered Flow** using SearchStep for structured retrieval workflows. +- **Control which fields are used for embeddings** to optimize search relevance. + +This tutorial demonstrates RAG using Oracle Database as a persistent, production-ready vector store. +You will use OracleDatabaseDatastore and Oracle AI Vector Search throughout. + + +Concepts shown in this guide +============================ + +- :ref:`VectorRetrieverConfig` and :ref:`SearchConfig` for configuring vector search +- :ref:`SearchToolBox` for providing search capabilities to Agents +- :ref:`SearchStep` for retrieval in Flows +- :ref:`VectorConfig` and :ref:`SerializerConfig` for controlling embedding generation +- :ref:`Embedding models` for converting text to vectors + +Before you begin, you must connect to Oracle Database and create the table(s) needed for vector search. See below. + +Step 0. OracleDatabaseDatastore: Connecting and Automated Table Preparation +=========================================================================== + +To use this guide, you should prepare an Oracle Database with vector search capability. This tutorial demonstrates an example for how you can automate the connection and table setup directly from Python. +To follow this guide, you just need to have a connection to Oracle Database and should be able to perform operations on the Database. + +**Connection & Authentication** + +The code automatically detects either mTLS or simple TLS database connectivity using environment variables. The following environment variables must be set for your Oracle connection: + +.. code-block:: shell + + # For mTLS connection (Autonomous DB/Wallet) + export ADB_CONFIG_DIR=encrypted/wallet/config + export ADB_WALLET_DIR=encrypted/wallet + export ADB_WALLET_SECRET='supersecret' + export ADB_DB_USER=garage_user + export ADB_DB_PASSWORD=secret + export ADB_DSN="adb....oraclecloud.com" + + # Or for TLS connection + export ADB_DB_USER=garage_user + export ADB_DB_PASSWORD=secret + export ADB_DSN="dbhost:port/servicename" + +Reference: `Oracle Database TLS setup guide `_ + +.. warning:: + Using environment variables for storing sensitive connection details is not suitable for production environments. + +The code will choose the most secure available connection automatically. + +**Table Schema Setup and DDL Execution** + +To be able to retrieve from you data, you need it stored in a database. We can use the Oracle Database to store the entities that will be retrieved using Oracle 23AI. +To connect to it, configure the client with ``oracledb`` and specify the schema of the data. +The schema for this example is: + +.. code-block:: sql + + CREATE TABLE motorcycles ( + owner_name VARCHAR2(255), + model_name VARCHAR2(255), + description VARCHAR2(255), + hp INTEGER, + serialized_text VARCHAR2(1023), + embeddings VECTOR + ); + +This schema includes both a conventional text representation (serialized_text) and a VECTOR column for semantic search. The Python code handles both the creation (and dropping) of this table and the population of its data. + +.. _setting_up: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Oracle-connection + :end-before: .. end-##_Oracle-connection + +The code: + +* Detects your connection configuration (mTLS/TLS) +* Creates the target table in Oracle using your credentials with: + * the table fields to match the entity schema (see next section) + * an additional `serial_text` TEXT field to store the string used for embeddings + * an additional `embeddings` VECTOR field for the vector search + +Note that if you already have a table configured with the same name, you will need to drop the table before running this code. +Refer to the :ref:`Cleaning Up section` to see how this can be done. + +**Required Privileges** + +Make sure your user has privileges to drop, create, insert, update, and select on the target table (motorcycles). + +Also make sure you have installed `oracledb `_. + +.. note:: + You can install the required package using pip: + + .. code-block:: shell + + pip install oracledb + +Setting Up RAG +============== + +A Retrieval-Augmented Generation (RAG) system is composed of two core components: a retriever and an LLM (Large Language Model). +The retriever is responsible for searching your data for relevant information, while the LLM uses the retriever as a tool to supplement its responses with up-to-date knowledge. +To achieve this, we would thus need both a retriever with an embedding model and an LLM to perform end-to-end RAG. + +Before creating these RAG-powered assistants, you will need to set up the data source which supports vector search capabilities. + +Step 1. Configure models +------------------------ + +You need an embedding model for the retriever as it converts your text data into embeddings (vector representations). +The retriever uses these embeddings to perform semantic searches, enabling the system to retrieve relevant information based on meaning rather than just keywords. + +Configure the embedding model for vector search: + + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Embedding-config + :end-before: .. end-##_Embedding-config + +Configure your LLM: + +The LLM (Large Language Model) plays two crucial roles in RAG. +First, it generates a suitable retrieval query to fetch relevant information from your datastore. +Then, after retrieval, the LLM formats and integrates the retrieved text into a coherent, user-facing response. +Understanding the role of the LLM is key to grasping why RAG involves both retrieval and generative capabilities: retrieval brings in up-to-date, +domain-specific knowledge, while the LLM ensures information is expressed in conversational form for the user. + +.. include:: ../_components/llm_config_tabs.rst + +Step 2. Define searchable data +------------------------------ + +First, define the schema for your data. Note that the collection and property names defined below should match the table and column names configured in Oracle Database (see the table we created in step 0). + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Entity-define + :end-before: .. end-##_Entity-define + +Next, we configure a vector and search config for searching in this data. A few things to note: + +* If you have configured a vector index, ensure you put the same distance metric in the ``distance_metric`` parameter of the :ref:`VectorRetrieverConfig`. Without doing so, the approximate search will not work. +* The embedding model passed in either the :ref:`VectorConfig` or the :ref:`VectorRetrieverConfig` should be the same as the model used to generate the corresponding embeddings column. If you specify an embedding model in both the classes, the embedding models must match. +* You can configure the ``vectors`` parameter in the :ref:`VectorRetrieverConfig` to explicitly specify the vector column or :ref:`VectorConfig` you want to search. +* If ``vectors`` is `None`, the vector column to search will be inferred by either an existing vector config with the same collection name or a vector column in the collection. If there are two or more matching vector configurations, an error will be raised. +* If you do not specify a ``collection_name`` in the :ref:`VectorConfig`, the config is applicable to all collections in your datastore. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Search-config + :end-before: .. end-##_Search-config + +Then, you can create the datastore with search capability by passing the search configuration. To fill the data, we perform serialization of fields using the :ref:`ConcatSerializerConfig`. +For each motorcycle entity, this will concatenate all fields and their values into a single string called ``serialized_text``, which is then embedded by the model. The resulting embedding vector is assigned to the ``embeddings`` field. +This approach gives you control over what text is represented in your vector index and is transparent/easy to audit. + +By default, all text fields in your entities are used to generate embeddings. However, you may want to exclude certain fields like IDs, prices, or metadata from the embedding calculation while still returning them in search results. +This can be achieved by configuring the ``columns_to_exclude`` parameter in :ref:`ConcatSerializerConfig`. + +Note that ``datastore.create()`` is used here for demonstration only and is not the recommended way to load data into Oracle Database tables. +For real applications, populate your tables with ``SQL`` (e.g., bulk INSERT/UPDATE), then use the Datastore APIs to index, search, and take advantage of WayFlow features. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Datastore-create-rag + :end-before: .. end-##_Datastore-create-rag + +**Create a Vector Index for Efficient Vector Search** + +For production semantic search, it is also recommended that you create a vector index on the embeddings field using Oracle's HNSW (or IVF) index. +Having a vector index configured is not necessary for search to work, but it will speed things up as it will use approximate search rather than using exact search. +Note that if you want to use the vector index as intended, the distance metric configured in the index should be the same as the distance metric used in the :ref:`VectorRetrieverConfig` (you can use +:ref:`SimilarityMetric` for simplicity). +The code below creates this index programmatically and commits it to your Oracle DB. (Skip this step for in-memory datastores.) + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Create-vector-index + :end-before: .. end-##_Create-vector-index + +You can test the search directly: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Direct-search-example + :end-before: .. end-##_Direct-search-example + + +With your RAG-ready datastore in place, the next step is to use it in real applications. In WayFlow, the two primary patterns for Retrieval-Augmented Generation are: + +- Integrating RAG capabilities into conversational Agents for dynamic, dialogue-driven retrieval +- Building Flows for more structured and predictable retrieval workflows. + +In the next sections, you'll see hands-on how to use both approaches, starting with Agents. + + +RAG in Agents +============= + +We'll start by showing how to empower your Agents with retrieval capabilities, allowing them to proactively fetch and reason over domain-specific information as part of their decision-making. +Agents provide a flexible approach to RAG by autonomously deciding when and how to search for information based on the conversation context. + +Step 1. Create search tools for the Agent +----------------------------------------- + +Convert your searchable datastore into tools that an Agent can use: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Agent_Tools_Rag + :end-before: .. end-##_Agent_Tools_Rag + +The ``get_search_tools`` method creates a :ref:`SearchToolBox` that: + +- Dynamically generates search tools for each collection +- Respects the ``k`` parameter to limit result count +- Returns results as JSON for easy parsing by the LLM + +Step 2. Create the RAG Agent +---------------------------- + +Create an Agent with search capabilities: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Agent_Create_Rag + :end-before: .. end-##_Agent_Create_Rag + +This Agent will: + +- Automatically use search tools when it needs information +- Combine search results with its reasoning capabilities +- Provide accurate answers based on your specific data + +Test the Agent: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Agent_Test_Rag + :end-before: .. end-##_Agent_Test_Rag + +The Agent autonomously decides when to search, what to search for, and how to use the results to answer questions. + + +RAG in Flows +============ + +While Agents offer flexibility, Flows provide a structured approach to RAG with predictable retrieval workflows ideal for specific use cases. + +Step 1. Create the RAG Flow +--------------------------- + +Create a Flow that searches for relevant information before generating a response: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Flow_Steps_Rag + :end-before: .. end-##_Flow_Steps_Rag + +Key points: + +- The :ref:`SearchStep` uses semantic search to find relevant documents based on the user's query. +- The ``k`` parameter limits the number of documents retrieved. +- Retrieved documents are passed to the LLM along with the original query for contextualized responses. + +Step 2. Build and test the Flow +------------------------------- + +Build the complete Flow with control and data connections: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Flow_Build_Rag + :end-before: .. end-##_Flow_Build_Rag + +The Flow provides a predictable pipeline: user input → search → response generation. + + +Advanced RAG Techniques +======================= + + +Filtering search results +------------------------ + +You can filter search results based on metadata: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Advanced_Filtering + :end-before: .. end-##_Advanced_Filtering + +Multiple search configurations +------------------------------ + +Create specialized search configurations for different use cases: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Advanced_Multi_Config + :end-before: .. end-##_Advanced_Multi_Config + +**How multiple search configs work:** + +- Each :ref:`SearchConfig` must have a unique name (auto-generated if not provided) +- Search configs can target the same collection with different settings (distance metrics, vector configs) +- Search Configs can also target multiple collections if no collection name is specified, provided there does not exist another search config which matches the collection name to search on. +- When calling :ref:`Datastore.search()`, you specify which config to use via the ``search_config`` parameter +- If no ``search_config`` is specified, the system looks for a default config for that collection, given that a ``collection_name`` is specified +- The first config that matches the collection (or has no specific collection) becomes the default + +**When to use each config** + +- ``precise_search``: Uses cosine similarity for semantic matching (best for meaning-based searches) +- ``broad_search``: Uses Euclidean distance for broader matches (considers all dimensions equally) +- You explicitly choose which to use: ``datastore.search(..., search_config="precise_search")`` + +Customizing search behavior in Agents +------------------------------------- + +Create specialized search toolboxes with different parameters: + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Advanced_Custom_Toolbox + :end-before: .. end-##_Advanced_Custom_Toolbox + +**How specialized toolboxes work:** + +- Each toolbox creates different search functions with fixed parameters +- ``detailed_search``: Always returns 10 results (k=10) for comprehensive analysis +- ``quick_search``: Always returns 1 result (k=1) for focused answers +- The Agent sees these as different tools: ``search_motorcycles_detailed`` vs ``search_motorcycles_quick`` + +**When each toolbox is used:** + +- The Agent autonomously decides based on: + + - The user's question complexity + - Instructions in ``custom_instruction`` + - Context of the conversation + +- For "tell me about all sport bikes" → likely uses ``detailed_search`` +- For "who owns the Vortex?" → likely uses ``quick_search`` +- The Agent's reasoning determines the choice, guided by your instructions + + +Manual Serialization of Fields for Embeddings +--------------------------------------------- + +In this example, we show a manual serialization approach that performs cross-field logic that cannot be expressed with :ref:`ConcatSerializerConfig`. +Instead of merely concatenating fields, we: +- Compute derived attributes (e.g., performance class and hp bands from numeric horsepower) +- Conditionally weight salient tokens (repeat model name for high-HP bikes) +- Inject domain keywords based on the description semantics +- Reorder fields and output a structured, sectioned Markdown document + +This goes beyond per-field preprocessing and simple separators; it uses the full entity structure at once and conditional logic across multiple fields. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :linenos: + :start-after: .. start-##_Manual_Serialization_Advanced + :end-before: .. end-##_Manual_Serialization_Advanced + +Example usage when generating embeddings: + +.. code-block:: python + + for entity in motorcycle_data: + entity["serialized_text"] = serialize_motorcycle_advanced(entity) + entity["embeddings"] = embedding_model.embed([entity["serialized_text"]])[0] + +**Why use explicit serialization?** + +- Cross-field logic: derive fields (e.g., performance class from hp) and conditionally add keywords. +- Conditional weighting: repeat or emphasize tokens under certain conditions (e.g., horsepower thresholds). +- Structured formatting: generate Markdown sections and control field ordering for domain salience. +- Auditable and deterministic: the exact text used for embeddings is transparent and reproducible. + +Limitations of ConcatSerializerConfig and when to choose manual serialization: + +- ConcatSerializerConfig is powerful for per-field concatenation with simple pre/post processing and exclusion of columns. +- It does not perform arbitrarily complex cross-field computations, conditional token weighting, or multi-field derived features. +- Choose manual serialization whenever you need entity-level reasoning to craft the embedding text, beyond simple concatenation and formatting. + +.. note:: + + Selective field embedding—using serializers to specify which fields participate in embedding generation—is best supported and straightforward in the :ref:`InMemoryDatastore` backend (see its API for serializer support). + For :ref:`OracleDatabaseDatastore`, you are responsible for constructing and storing the embeddings explicitly, and there is no out-of-the-box field-level selection. + For configuring the serialized text and embeddings column externally, you can make use of :ref:`ConcatSerializerConfig` + outside the Datastore while generating the serialized text for the embeddings. :ref:`OracleDatabaseDatastore` assumes that the embedding column has already been generated and does not implicitly create embeddings. + +.. note:: + + For rapid prototyping, use :ref:`InMemoryDatastore` with custom serializers for full flexibility, then migrate to :ref:`OracleDatabaseDatastore` for production workloads that require persistence and scalability. + + +Agent Spec Exporting/Loading +============================ + +You can export the agent configuration to its Agent Spec configuration using the :ref:`AgentSpecExporter`. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_rag.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_rag.yaml + :language: yaml + +.. warning:: + + The Oracle Database Connection Config objects contain several sensitive values + (like username, password, wallet location) that will not be serialized by the ``AgentSpecExporter``. + These will be serialized as references that must be resolved at loading time, by specifying the values + of these sensitive fields in the ``component_registry`` argument of the loader: + + .. literalinclude:: ../code_examples/howto_rag.py + :language: python + :start-after: .. start-##_Provide_sensitive_information_when_loading_the_Agent_Spec_config + :end-before: .. end-##_Provide_sensitive_information_when_loading_the_Agent_Spec_config + +You can then load the configuration back to an assistant using the :ref:`AgentSpecLoader`. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - :ref:`PluginToolFromToolBox` + - :ref:`PluginSearchToolBox` + - :ref:`PluginOracleDatabaseDatastore` + - :ref:`PluginSearchConfig` + - :ref:`PluginVectorRetrieverConfig` + - :ref:`PluginVllmEmbeddingConfig` + - :ref:`PluginPromptTemplate` + - :ref:`PluginJsonToolOutputParser` + - :ref:`PluginRemoveEmptyNonUserMessageTransform` + - :ref:`PluginCoalesceSystemMessagesTransform` + - :ref:`PluginLlamaMergeToolRequestAndCallsTransform` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + +.. _cleanup: + +Cleaning Up Datastore +===================== + +Before moving on, you may want to cleanup the table created in Oracle Database for this tutorial. For cleaning up, you can use the following code below. +This code will drop the ``motorcycles`` from your Oracle Database using the ``environment_config`` function defined in the :ref:`Setting Up` section. + +.. literalinclude:: ../code_examples/howto_rag.py + :language: python + :start-after: .. start-##_Cleanup_datastore + :end-before: .. end-##_Cleanup_datastore + +Recap +===== + +In this guide, you learned how to build RAG-powered assistants using WayFlow: + + +The key difference between Agents and Flows for RAG: + +- **Agents** offer dynamic, autonomous retrieval based on the conversation context - ideal when you want the AI to decide when and what to search +- **Flows** provide predictable, structured retrieval workflows - ideal when you want consistent behavior for specific use cases + +Key techniques covered: + +- **Basic RAG**: Using all fields for embeddings and search +- **Filtered search**: Limiting results based on metadata +- **Multiple search configs**: Different strategies for different use cases with explicit selection +- **Multiple toolboxes**: Allowing Agents to choose between different search strategies autonomously + + +.. important:: + **Before deploying your RAG application to production, you MUST:** + + 1. **Configure Oracle AI Vector Search** for scalable vector operations + 2. **Test performance** with production-scale data + 3. **Implement proper error handling** and monitoring + + For development and testing, you can use the ``InMemoryDataStore``, the same APIs work with both datastores: + + .. code-block:: python + + # Development (NOT for production) + datastore = InMemoryDatastore(schema={"motorcycles": motorcycles}) + + # Production (use this instead) + datastore = OracleDatabaseDatastore( + connection_string="your_oracle_connection", + schema={"motorcycles": motorcycles} + # connection db params + ) + + See the `OracleDatabaseDatastore guide` for complete migration instructions. + + +Next steps +========== + +Deployment Considerations: Now your application is backed by OracleDatabaseDatastore from the start. +Your setup is production-ready, persistent, and scalable using Oracle AI Vector Search. + +- Always test with your own database connection and schema for production. +- Ensure your Oracle user has all necessary table privileges. +- For advanced vector functionality, see the OracleDatabaseDatastore API guide. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_rag.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_remote_tool_expired_token.rst.txt b/26.1.2/_sources/core/howtoguides/howto_remote_tool_expired_token.rst.txt new file mode 100644 index 000000000..4a95a3069 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_remote_tool_expired_token.rst.txt @@ -0,0 +1,199 @@ +=========================================================== +How to Do Remote API Calls with Potentially Expiring Tokens +=========================================================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_remote_tool_expired_token.py + :link-alt: MCP how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + This guide assumes familiarity with: + + - :doc:`Building Assistants with Tools ` + +When building assistants with tools that reply on remote API calls, it is important to handle the authentication failures gracefully—especially those caused by expired access tokens. +In this guide, you will build an assistant that calls a mock service requiring a valid token for authentication. + +Setup +===== + +To demonstrate the concept in a safe environment, we first set up a local mock API server (here, using FastAPI). +This simulates an endpoint that requires and validates an authentication token. If the token provided is: + +* a valid token (`valid-token`): the service responds with a success message. +* an expired token (`expired-token`) or invalid token: the `401 Unauthorized` error is returned with details. + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Mock_server + :end-before: .. end-##_Mock_server + :linenos: + +Basic implementation +==================== + +In this example, you will build a simple :ref:`Agent ` that includes a :ref:`Flow ` with three steps: + +* A start step to get the user name +* A step to trigger a client tool that collects a token from the user +* A step to call a remote API given the user name and the token + +This guide requires the use of an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +Importing libraries +------------------- +First import what is needed for this tutorial: + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Import_libraries + :end-before: .. end-##_Import_libraries + :linenos: + +Creating the steps +------------------ +Define the variable names and steps. + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Variable_names + :end-before: .. end-##_Variable_names + :linenos: + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Defining_steps + :end-before: .. end-##_Defining_steps + :linenos: + +In this simple example, we manually input the user name and the token in the code. +For a more interactive approach, consider using :ref:`InputMessageStep ` to prompt the user to enter these values during execution. + +Creating the flow +----------------- + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Defining_flow + :end-before: .. end-##_Defining_flow + :linenos: + +This flow simply proceeds through three steps as defined in the ``control_flow_edges``. +The ``data_flow_edges`` connect the outputs of each step—the user name from ``start_step`` and the token from ``get_token_tool_step``—to the inputs required by ``call_api_step``. + +Testing the flow +---------------- + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Testing_flow + :end-before: .. end-##_Testing_flow + :linenos: + +To simulate a valid user, provide ``auth_token = "valid-token"``. +To test expiry handling, use ``auth_token = "expired-token"``, which is expected to raise an error. +The flow should pause at the token step, mimicking a credential input prompt, then proceed upon receiving input. + +Creating an agent +----------------- +Now, create an agent that includes the defined flow: + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Defining_agent + :end-before: .. end-##_Defining_agent + :linenos: + +Testing the agent +----------------- + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Testing_agent + :end-before: .. end-##_Testing_agent + :linenos: + +The code block above demonstrates an interaction flow between a user and the agent, simulating how the assistant processes a remote, authenticated API call. +During the first execution, the agent determines that a token is required and issues a tool request to the client. This is reflected by the ``status`` being an instance of ``ToolRequestStatus``. +After the client provides the required credential (the token), the second execution resumes the conversation. +If authentication is successful, the agent proceeds to call the API, processes the response, and generates a user message as its reply. +At this stage, the ``status`` should be ``UserMessageRequestStatus``, which indicates that the agent has completed processing and is now ready to present a message to the user or wait for the next user prompt. +Checking for ``UserMessageRequestStatus`` ensures that your code only tries to access the assistant's reply when it is actually available. + + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_remote_tool_expired_token.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_remote_tool_expired_token.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_remote_tool_expired_token.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginPromptTemplate`` + - ``PluginRemoveEmptyNonUserMessageTransform`` + - ``ExtendedToolNode`` + - ``ExtendedAgent`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + + +Next steps +========== + +In this guide, you learned how to define a simple flow that retrieves a token from the user and uses it to authenticate remote API calls. +To continue learning, checkout: + +- :doc:`How to Catch Exceptions in Flows `. + +Full code +========= + +.. literalinclude:: ../end_to_end_code_examples/howto_remote_tool_expired_token.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_retry_configuration.rst.txt b/26.1.2/_sources/core/howtoguides/howto_retry_configuration.rst.txt new file mode 100644 index 000000000..1d75eafb4 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_retry_configuration.rst.txt @@ -0,0 +1,120 @@ +.. _top-howtoretryconfiguration: + +====================================================== +How to Configure Retries on LLMs and Remote Components +====================================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_retry_configuration.py + :link-alt: Retry configuration how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`LLM configuration ` + - :doc:`Using agents ` + - :doc:`Doing remote API calls ` + +WayFlow remote components now share the same :ref:`RetryPolicy ` object. +You can use it to configure retry timing and per-attempt request timeouts consistently across +:ref:`LLMs `, :ref:`embedding models `, +:ref:`ApiCallStep `, :ref:`RemoteTool `, +remote MCP transports such as :ref:`StreamableHTTPTransport `, +:ref:`OciAgent `, and :ref:`A2AAgent `. + + +Create a Retry Policy +===================== + +Start by defining a ``RetryPolicy`` with the retry behavior you want to reuse. + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Create_a_retry_policy + :end-before: .. end-##_Create_a_retry_policy + + +Apply the Policy to Remote Components +===================================== + +You can then pass the same retry policy to any supported remote component. + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Configure_a_remote_LLM + :end-before: .. end-##_Configure_a_remote_LLM + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Configure_remote_steps_and_tools + :end-before: .. end-##_Configure_remote_steps_and_tools + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Configure_remote_agents_and_MCP_transports + :end-before: .. end-##_Configure_remote_agents_and_MCP_transports + +WayFlow applies ``request_timeout`` per attempt. Retries are limited to transient failures such +as configured recoverable status codes, eligible ``5xx`` responses, and connection errors. +Authentication failures, validation failures, and TLS/certificate verification failures are not retried. + + +Agent Spec Exporting/Loading +============================ + +You can export a configuration that includes the retry policy to Agent Spec using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_retry_configuration.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_retry_configuration.yaml + :language: yaml + +You can then load the configuration back with the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_retry_configuration.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + +Next steps +========== + +Now that you have learned how to configure retries on remote components, you may proceed to +:doc:`How to Use OCI Generative AI Agents ` or +:doc:`How to Connect to A2A Agents `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_retry_configuration.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_serdeser.rst.txt b/26.1.2/_sources/core/howtoguides/howto_serdeser.rst.txt new file mode 100644 index 000000000..ad4a59dbf --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_serdeser.rst.txt @@ -0,0 +1,225 @@ +.. _top-serdeser: + +================================================= +How to Serialize and Deserialize Flows and Agents +================================================= + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_serdeser.py + :link-alt: Serialization and deserialization how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Agents <../tutorials/basic_agent>` + +Assistant serialization is a crucial feature in WayFlow that allows you to save and load :ref:`Agents ` and :ref:`Flows `, +making it easy to persist their configurations and reuse them as needed. + +In this guide, you will learn how to: + +- Serialize a simple Agent or Flow and deserialize it back into an executable assistant. +- Use serialization for more complex assistants using tools. + +.. image:: ../_static/howto/ser_deser.svg + :align: center + :scale: 100% + :alt: Serialization/deserialization of Agents and Flows in WayFlow + + +Saving and loading simple assistants +==================================== + +This section shows you how to serialize and reload WayFlow :ref:`Agents ` and :ref:`Flows `. + + +Flows +^^^^^^ + +Start by creating a simple ``Flow`` that takes a user question as input and responds using an LLM. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Flow_Creation + :end-before: .. end-##_Simple_Flow_Creation + +**API Reference:** :ref:`Flow ` | :ref:`PromptExecutionStep ` + +Once you have built the flow, you can serialize it using the ``serialize`` helper function. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Flow_Serialization + :end-before: .. end-##_Simple_Flow_Serialization + +**API Reference:** :ref:`serialize ` + +Then, save the serialized flow as a regular text file. + +To deserialize the flow configuration back, use the ``autodeserialize`` helper function. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Flow_Deserialization + :end-before: .. end-##_Simple_Flow_Deserialization + +**API Reference:** :ref:`autodeserialize ` + +After deserialization, the flow is ready to execute like any other WayFlow assistant. + +.. note:: + The serialized configuration file contains all elements that compose the :ref:`Flow `. + However, this file is not intended to be human-readable and should only be handled using the ``serialize`` and ``autodeserialize`` functions. + + +Agents +^^^^^^ + +Continue to building a simple conversational ``Agent`` that can answer user questions. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Agent_Creation + :end-before: .. end-##_Simple_Agent_Creation + +**API Reference:** :ref:`Agent ` + +Once you have built the agent, you can serialize it using the ``serialize`` helper function. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Agent_Serialization + :end-before: .. end-##_Simple_Agent_Serialization + +**API Reference:** :ref:`serialize ` + +Then, save the serialized agent as a regular text file. + +To deserialize the agent configuration back, use the ``autodeserialize`` helper function. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Simple_Agent_Deserialization + :end-before: .. end-##_Simple_Agent_Deserialization + +**API Reference:** :ref:`autodeserialize ` + +Similar to the Flow example above, once deserialized, the agent is ready to execute like any other WayFlow assistant. + + +Saving and loading assistants equipped with tools +================================================= + +In this more advanced example, you will build assistants that use WayFlow Tools (such as :ref:`ServerTool `). +These assistants require additional code to deserialize them into executable assistants. + + +Flows +^^^^^^ + +Create a ``Flow`` that asks the user for an input text, counts the number of characters, and generates a message with the result. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Complex_Flow_Creation + :end-before: .. end-##_Complex_Flow_Creation + +**API Reference:** :ref:`InputMessageStep ` | +:ref:`OutputMessageStep ` | :ref:`ToolExecutionStep ` | +:ref:`ServerTool ` + +Serialize your flow just like any other assistant. + +To deserialize the flow, you need to provide context about the tool used in the original flow. +This can be done using ``DeserializationContext``. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Complex_Flow_Deserialization + :end-before: .. end-##_Complex_Flow_Deserialization + +After registering the tool in the dictionary of tools, pass the deserialization context to the ``autodeserialize`` function to deserialize the flow. + +.. important:: + Ensure that tool names in ``DeserializationContext.registered_tools`` are unique to avoid conflicts. + + +Agents +^^^^^^ + +Create an ``Agent`` that can access a tool to count the number of characters in a given text (this agent is equivalent to the flow example above). + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Complex_Agent_Creation + :end-before: .. end-##_Complex_Agent_Creation + +**API Reference:** :ref:`ServerTool ` + +Serialize your agent just like any other assistant. + +Similar to the Flow example, deserializing the agent requires providing context about the tool used in the original agent. +This can be done using ``DeserializationContext``. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Complex_Agent_Deserialization + :end-before: .. end-##_Complex_Agent_Deserialization + +After registering the tool in the dictionary of tools, pass the deserialization context to the ``autodeserialize`` function to deserialize the agent. + +.. important:: + Ensure that tool names in ``DeserializationContext.registered_tools`` are unique to avoid conflicts. + + +Agent Spec Exporting/Loading +============================ + +You can export the flow or agent configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + +And load it back using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_serdeser.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + + +Recap +===== + +In this guide, you learned how to serialize WayFlow :ref:`Agents ` and :ref:`Flows `, as well as how to handle deserialization for assistants that use tools. + + +Next steps +========== + +Having learned how to serialize and deserialize assistants built with WayFlow, you may now proceed to: + +- :doc:`How to Create Conditional Transitions in Flows ` +- :doc:`How to Create a ServerTool from a Flow ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_serdeser.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_serialize_conversations.rst.txt b/26.1.2/_sources/core/howtoguides/howto_serialize_conversations.rst.txt new file mode 100644 index 000000000..9d3625209 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_serialize_conversations.rst.txt @@ -0,0 +1,296 @@ +.. _top-howtoserializeconversations: + +============================================== +How to Serialize and Deserialize Conversations +============================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_serialize_conversations.py + :link-alt: Serialize conversation how-to script + + Python script/notebook for this guide. + + + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Flows <../tutorials/basic_flow>` + +When building AI assistants, it id=s often necessary to save the state of a conversation to disk and restore it later. +This is essential for creating persistent applications that can: + +- Resume conversations after an application restart +- Save user sessions for later continuation +- Implement conversation history and analytics +- Support multi-session workflows + +In this tutorial, you will learn how to: + +- **Serialize Agent conversations** to JSON files +- **Serialize Flow conversations** at any point during execution +- **Deserialize and resume** both types of conversations +- **Build persistent conversation loops** that survive application restarts + +Concepts shown in this guide +============================ + +- :ref:`serialize ` to convert conversations to storable format +- :ref:`autodeserialize ` to restore conversations from storage +- Handling conversation state persistence for both Agents and Flows + +Basic Serialization +=================== + +Step 1. Add imports and configure LLM +------------------------------------- + +Start by importing the necessary packages for serialization: + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Imports_for_this_guide + :end-before: .. end-##_Imports_for_this_guide + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +Step 2. Create storage functions +-------------------------------- + +Define helper functions to store and load conversations: + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Create_storage_functions + :end-before: .. end-##_Create_storage_functions + +These functions: + +- Use WayFlow's ``serialize()`` to convert conversations to a storable format +- Store multiple conversations in a single JSON file indexed by conversation ID +- Use ``autodeserialize()`` to restore the original conversation objects + +Serializing Agent conversations +=============================== + +Agent conversations can be serialized at any point during execution: + +Step 1. Create an Agent +----------------------- + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Creating_an_agent + :end-before: .. end-##_Creating_an_agent + +Step 2. Run the conversation +---------------------------- + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Run_the_agent + :end-before: .. end-##_Run_the_agent + + +Step 3. Serialize the conversation +---------------------------------- + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Serialize_the_conversation + :end-before: .. end-##_Serialize_the_conversation + +Step 4. Deserialize the conversation +------------------------------------ + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Deserialize_the_conversation + :end-before: .. end-##_Deserialize_the_conversation + + +Key points: + +- Each conversation has a unique ``conversation_id`` +- The entire conversation state is preserved, including message history +- Loaded conversations retain their complete state and can resume execution +- Access messages through ``conversation.message_list.messages`` + +Serializing Flow Conversations +============================== + +Flow conversations require special attention as they can be serialized mid-execution: + +Step 1. Create a Flow +--------------------- + +First, create a flow using the builder function. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Creating_a_flow + :end-before: .. end-##_Creating_a_flow + +Step 2. Run the conversation +---------------------------- + +Then start and run the flow conversation. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Run_the_flow + :end-before: .. end-##_Run_the_flow + + +Step 3. Serialize during execution +---------------------------------- + +You can now serialize the conversation during its execution, for instance +here the flow is requesting the user to input some information but you can +serialize the conversation and resume it later. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Serialize_before_providing_user_input + :end-before: .. end-##_Serialize_before_providing_user_input + + +Step 4. Deserialize the conversation +------------------------------------ + +You can now load back the serialized conversation. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Deserialize_the_flow_conversation + :end-before: .. end-##_Deserialize_the_flow_conversation + +Step 5. Resume the conversation execution +----------------------------------------- + +You can resume the conversation from its state before serializing it. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Resume_the_conversation_execution + :end-before: .. end-##_Resume_the_conversation_execution + + +Important considerations: + +- Flows can be serialized while waiting for user input +- The loaded Flow conversation resumes exactly where it left off +- User input can be provided to the loaded conversation to continue execution +- If your component uses :ref:`MessageSummarizationTransform ` or :ref:`ConversationSummarizationTransform `, their configuration is serialized with the component and restored on deserialization +- If those transforms use an :ref:`InMemoryDatastore ` for caching, cached summary rows are not preserved across restarts; use a persistent datastore if cache continuity matters + +Building persistent applications +================================ + +For real-world applications, you'll want to create persistent conversation loops: + + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Creating_a_persistent_conversation_loop + :end-before: .. end-##_Creating_a_persistent_conversation_loop + +This function: + +- Loads existing conversations or starts new ones +- Saves state before waiting for user input +- Allows users to exit and resume later +- Returns the conversation ID for future reference + +Best Practices +============== + +1. **Save before user input**: Always serialize conversations before waiting for user input to prevent data loss. + +2. **Use unique IDs**: Store conversations using their built-in ``conversation_id`` to avoid conflicts. + +3. **Handle errors gracefully**: Wrap deserialization in try-except blocks to handle missing or corrupted data. + +4. **Consider storage format**: While JSON is human-readable, consider other formats for production use. + +5. **Version your serialization**: Consider adding version information to handle future schema changes. + +Limitations +=========== + +- **Tool state**: When using tools with Agents, ensure tools are stateless or their state is managed separately. +- **Large conversations**: Very long conversations may result in large serialized files. +- **Binary data**: The default JSON serialization does not handle binary data directly. + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_serialize_conversations.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_serialize_conversations.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_serialize_conversations.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + + + +Next steps +========== + +In this guide, you learned how to: + +- Serialize both Agent and Flow conversations +- Restore conversations and continue execution +- Build persistent conversation loops +- Handle conversation state across application restarts + +Having learned how to serialize a conversation, you may now proceed to :doc:`How to Serialize and Deserialize Flows and Agents `. + +- :doc:`Serialization ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_serialize_conversations.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_serve_agents.rst.txt b/26.1.2/_sources/core/howtoguides/howto_serve_agents.rst.txt new file mode 100644 index 000000000..5c348e917 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_serve_agents.rst.txt @@ -0,0 +1,182 @@ +.. _top-howtoserveagents: + +================================ +How to Serve Agents with WayFlow +================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_serve_agents.py + :link-alt: Serve Agents how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Datastores <../howtoguides/howto_datastores>` + - :doc:`WayFlow Tools <../api/tools>` + +WayFlow can host agents behind an +`OpenAI Responses API `_ compatible +endpoint. Reliable serving unlocks predictable SLAs, reusable state, and consistent security, while +letting clients keep using familiar OpenAI SDKs. Start with an in-memory setup for quick +experiments, then add persistence to reuse conversation state and layer FastAPI security controls +that fit your environment. + + +Create an agent to host +======================= + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +.. note:: + API keys should not be stored anywhere in the code. Use environment variables or tools such as `python-dotenv `_. + + +Then, create or reuse an agent you want to serve. You can define it as code: + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Define_the_agent + :end-before: .. end-##_Define_the_agent + +API Reference: :ref:`Agent ` | :ref:`Tool ` + + +Export and reload agent specs +============================= + +Save your agent as an Agent Spec so you can deploy from a config file or ship it to another team. +Reloading requires a ``tool_registry`` that maps tool names back to callables. + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Export_agent_spec + :end-before: .. end-##_Export_agent_spec + +API Reference: :ref:`AgentSpecExporter ` | :ref:`AgentSpecLoader ` + + +Run an in-memory Responses API server +===================================== + +Expose the agent with :ref:`OpenAIResponsesServer `. The server mounts +``/v1/responses`` and ``/v1/models`` endpoints that work with the official ``openai`` SDK or +:ref:`OpenAICompatibleModel `. + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Serve_in_memory + :end-before: .. end-##_Serve_in_memory + +You can now call the server with an OpenAI-compatible client: + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Call_the_server + :end-before: .. end-##_Call_the_server + +API Reference: :ref:`OpenAIResponsesServer ` + + +Persist conversations with datastores +===================================== + +To reuse conversation history across requests or server restarts, attach a datastore. Use +:ref:`ServerStorageConfig ` to define table and column names, then pass a +supported :ref:`Datastore ` implementation such as +:ref:`PostgresDatabaseDatastore ` or +:ref:`OracleDatabaseDatastore `. + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Persistent_storage + :end-before: .. end-##_Persistent_storage + +In production, create the table beforehand or run ``wayflow serve`` with ``--setup-datastore yes`` +to let WayFlow prepare it when the backend supports schema management. It will not override any +existing table, so you will need to first delete any existing table to allow wayflow to set it up +for you. + +API Reference: :ref:`ServerStorageConfig ` | :ref:`Datastore ` + + +Add FastAPI security controls +============================= + +:ref:`OpenAIResponsesServer ` gives you the FastAPI ``app`` instance so +you can stack your own middleware, dependencies, or routers. This example enforces a simple bearer +token check. + +.. literalinclude:: ../code_examples/howto_serve_agents.py + :language: python + :start-after: .. start-##_Add_fastapi_security + :end-before: .. end-##_Add_fastapi_security + +Replace the token check with your own authentication handler (OAuth2, mTLS validation, signed +cookies, IP filtering, etc.) and add rate limiting or CORS rules as needed. + +API Reference: :ref:`OpenAIResponsesServer ` + + +Use the CLI +=========== + +You can also serve an agent spec file directly from the CLI: + +.. code-block:: bash + + wayflow serve \ + --api openai-responses \ + --agent-config hr_agent.json \ + --agent-id hr-assistant \ + --server-storage postgres-db \ + --datastore-connection-config postgres_conn.yaml \ + --setup-datastore yes + + +Pass ``--tool-registry`` to load your own tools, swap ``--server-storage`` to ``oracle-db`` or +``in-memory``, and set ``--server-storage-config`` to override column names. See the :ref:`API reference ` for a +complete description of all arguments. + +.. warning:: + This CLI does not implement any security features; use it only for development or inside an + already-secured environment such as OCI agent deployments. Missing controls include: + + - **Authentication**: No verification of caller identity—anyone with network access can invoke the agent. + - **Authorization**: No role or permission checks to restrict which users can access specific agents or actions. + - **Rate limiting**: No protection against excessive requests that could exhaust resources or incur runaway costs. + - **TLS/HTTPS**: Traffic is unencrypted by default, risking interception of sensitive prompts and responses. + + For production deployments, wrap the server with an API gateway, reverse proxy, or custom FastAPI middleware that enforces these controls. + + + +Next steps +========== + +- :doc:`Use Agents in Flows ` +- :doc:`Connect Assistants to Your Data ` +- :doc:`Build Assistants with Tools ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code +for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_serve_agents.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_swarm.rst.txt b/26.1.2/_sources/core/howtoguides/howto_swarm.rst.txt new file mode 100644 index 000000000..5ff056c43 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_swarm.rst.txt @@ -0,0 +1,419 @@ +.. _top-howtoswarm: + +============================== +How to Build a Swarm of Agents +============================== + + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_swarm.py + :link-alt: Swarm how-to script + + Python script/notebook for this guide. + + + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Agents <../tutorials/basic_agent>`. + +The Swarm pattern is a type of agentic pattern that takes inspiration from `Swarm intelligence `_, +a phenomenon commonly seen in ant colonies, bee hives, and bird flocks, where coordinated behavior emerges from many simple actors rather than a central controller. +In this agentic pattern, each agent is assigned a specific responsibility and can delegate tasks to other specialized agents to improve overall performance. + + +**When to use the Swarm pattern?** + +Compared to using a :doc:`hierarchical multi-agent pattern `, the communication in :ref:`Swarm ` pattern reduces the number of LLM calls +as showcased in the diagram below. + +.. image:: ../_static/howto/hierarchical_vs_swarm.svg + :align: center + :scale: 70% + :alt: How the Swarm pattern compares to hierarchical multi-agent pattern + + + +In the **hierarchical pattern**, a route User → Agent K → User will require: + +1. All intermediate agent to call the correct sub-agent to go down to the Agent K. +2. The Agent K to generate its answer. +3. All intermediate agents to relay the answer back to the user. + +In the **swarm pattern**, a route User → Agent K → User will require: + +1. The first agent to call or handoff the conversation the Agent K (provided that the developer allows the connection between the two agents). +2. The Agent K to generate its answer. +3. The first agent to relay the answer (only when NOT using handoff; with handoff the Agent K **replaces** the first agent and is thus directly communicating with the human user) + +------- + + +This guide presents an example of a simple Swarm of agents applied to a medical use case. + +.. image:: ../_static/howto/swarm_example.svg + :align: center + :scale: 90% + :alt: Example of a Swarm agent pattern for medical application + +This guide will walk you through the following steps: + +1. Defining agents equipped with tools +2. Assembling a Swarm using the defined agents +3. Executing the Swarm of agents + +It also covers how to enable ``handoff`` when building the ``Swarm``. + + +.. warning:: + + The ``Swarm`` agentic pattern is currently in beta (e.g., it cannot yet be used in a ``Flow``). + Its API and behavior are not guaranteed to be stable and may evolve in future versions. + +For more information about ``Swarm`` and other agentic patterns in WayFlow, contact the AgentSpec development team. + + + +Basic implementation +==================== + +First import what is needed for this guide: + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :linenos: + :start-after: .. start-##_Imports_for_this_guide + :end-before: .. end-##_Imports_for_this_guide + + +To follow this guide, you will need access to a large language model (LLM). +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +In this section, you will define the agents that will later be used to build the Swarm of agents. + + + + +Creating the tools +------------------ + +The Swarm in this example consists of three :ref:`Agents `, each equipped with a single :ref:`Tool `. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Creating_the_tools + :end-before: .. end-##_Creating_the_tools + +**API Reference:** :ref:`tool ` + + +Defining the agents +------------------- + +The three agents need to be given the following elements: + +* A name +* A description +* A system prompt (the instruction to give to the LLM to solve a given task) +* A LLM +* Some optional tools + + +General Practitioner Agent +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The first agent the user interacts with is the General Practitioner Agent. + +This agent is equipped with the symptoms checker tool, and can interact with the **Pharmacist Agent** +as well as the **Dermatologist Agent**. + + +.. collapse:: Prompt for the General Practitioner Agent + + .. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Prompt_for_the_General_Practitioner_Agent + :end-before: .. end-##_Prompt_for_the_General_Practitioner_Agent + + +The General Practitioner Agent can be configured as follows: + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Define_the_General_Practitioner_Agent + :end-before: .. end-##_Define_the_General_Practitioner_Agent + + + +Pharmacist Agent +^^^^^^^^^^^^^^^^ + +The Pharmacist Agent is equipped with the tool to obtain medication information. +This agent cannot initiate a discussion with the other agents in the Swarm. + +.. collapse:: Prompt for the Pharmacist Agent + + .. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Prompt_for_the_Pharmacist_Agent + :end-before: .. end-##_Prompt_for_the_Pharmacist_Agent + + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Define_the_Pharmacist_Agent + :end-before: .. end-##_Define_the_Pharmacist_Agent + + + +Dermatologist Agent +^^^^^^^^^^^^^^^^^^^ + +The final agent in the Swarm is the Dermatologist agent which is equipped with a tool to query a skin condition knowledge base. +This agent can initiate a discussion with the **Pharmacist Agent**. + + +.. collapse:: Prompt for the Dermatologist Agent + + .. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Prompt_for_the_Dermatologist_Agent + :end-before: .. end-##_Prompt_for_the_Dermatologist_Agent + + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Define_the_Dermatologist_Agent + :end-before: .. end-##_Define_the_Dermatologist_Agent + + +Creating the Swarm +------------------ + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Creating_the_Swarm + :end-before: .. end-##_Creating_the_Swarm + +**API Reference:** :ref:`Swarm ` + +The Swarm has two main parameters: + +- The ``first_agent`` — the initial agent the user interacts with (in this example, the General Practitioner Agent). +- A list of relationships between agents. + +The ``relationships`` parameter defines the communication edges between agents. +Each relationship is expressed as a tuple of Caller Agent → Recipient Agent, indicating that the caller is permitted to delegate work to the recipient. + +These relationships apply to both types of delegation supported by the Swarm: + +- They determine which agents may contact another agent using the send-message tool. +- They also define which pairs of agents are eligible for a handoff, meaning the full user–agent conversation can be transferred from the caller to the recipient. + +In this example, the General Practitioner Doctor Agent can delegate to both the Pharmacist and the Dermatologist. +The Dermatologist can also delegate with the Pharmacist. + +When invoked, each agent can either respond to its caller (a human user or another agent) or choose to initiate a discussion with +another agent if they are given the capability to do so. + +You can also attach message transforms directly on the ``Swarm`` with the ``transforms`` parameter. This is useful for patterns such as +:ref:`MessageSummarizationTransform ` and :ref:`ConversationSummarizationTransform `, +where you want long-context handling to apply to the active agent's effective runtime prompt inside the swarm. +These swarm-level transforms are prepended ahead of any transforms already configured on the ``swarm_template``. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Creating_the_Swarm_With_Transforms + :end-before: .. end-##_Creating_the_Swarm_With_Transforms + +.. _swarm_transform_ordering: + +.. note:: + **How ``transforms`` and ``swarm_template`` work together** + + During execution, WayFlow always renders prompts from the configured ``swarm_template``, and applies that prompt surface to the active agent. + The active agent's own pre-rendering transforms are prepended first, then ``swarm.transforms``, + and finally any pre-rendering transforms already configured directly on ``swarm_template``. + The effective pre-rendering order is therefore: + + 1. ``active_agent.transforms`` + 2. ``swarm.transforms`` + 3. ``swarm_template.pre_rendering_transforms`` + + This means the most specific transforms run first: agent-level before swarm-level, and swarm-level before + template-level transforms. + + If you use ``_DEFAULT_SWARM_CHAT_TEMPLATE``, note that its built-in formatting transforms are + ``post_rendering_transforms``. They run after the pre-rendering order above and after the template has been rendered. + +If you serialize and later deserialize a swarm conversation, the summarization transform configuration is restored as part of the prompt template and component configuration. +For persistent deployments, prefer a database-backed datastore for the summary cache so previously computed summaries remain available after restarts. + +Executing the Swarm +------------------- + +Now that the Swarm is defined, you can execute it using an example user query. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Running_the_Swarm + :end-before: .. end-##_Running_the_Swarm + +We recommend to implement an :ref:`execution loop ` to execute the ``Swarm``, such as the following: + + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :linenos: + :start-after: .. start-##_Running_with_the_execution_loop + :end-before: .. end-##_Running_with_the_execution_loop + + +Advanced usage +============== + +Different handoff modes in Swarm +-------------------------------- + +One of the key benefits of Swarm is its handoff mechanism. When enabled, an agent can hand off its conversation — that is, transfer the entire message history between it and the human user — to another agent within the Swarm. + +The handoff mechanism helps reduce response latency. +Normally, when agents talk to each other, the "distance" between the human user and the final answering agent increases, because messages must travel through multiple agents. +Handoff avoids this by directly transferring the conversation to another agent: while the agent changes, the user's distance to the active agent stays the same. + +Swarm provides three handoff modes: + +- ``HandoffMode.NEVER`` + Disables the handoff mechanism. Agents can still communicate with each other, but cannot transfer the entire user conversation to another agent. + In this mode, the ``first_agent`` is the only agent that can directly interact with the human user. + +- ``HandoffMode.OPTIONAL`` (default) + Agents may perform a handoff when it is beneficial. + A handoff is performed only when the receiving agent is able to take over and independently address the user’s request. + This strikes a balance between multi-agent collaboration and minimizing overhead. + +- ``HandoffMode.ALWAYS`` + Agents must use the handoff mechanism whenever delegating work to another agent. Direct *send-message* tools between agents are disabled in this mode. + This mode is useful when most user requests are best handled by a single expert agent rather than through multi-agent collaboration. + + +To set the handoff mode, simply use :ref:`HandoffMode ` and select the desired mode. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Enabling_handoff_in_the_Swarm + :end-before: .. end-##_Enabling_handoff_in_the_Swarm + +**API Reference:** :ref:`HandoffMode ` + +Agent Spec Exporting/Loading +---------------------------- + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_swarm.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_swarm.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +Using Swarm within a Flow +========================= + +The ``Swarm`` pattern can be integrated into a :ref:`Flow ` using the :ref:`AgentExecutionStep `. + +Here's an example of how to integrate a swarm into a flow: + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Using_Swarm_within_a_Flow + :end-before: .. end-##_Using_Swarm_within_a_Flow + +You can run the flow with: + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Run_Swarm_within_a_Flow + :end-before: .. end-##_Run_Swarm_within_a_Flow + +Agent Spec Exporting/Loading +---------------------------- + +You can export the flow configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec2 + :end-before: .. end-##_Export_config_to_Agent_Spec2 + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_swarm2.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_swarm2.yaml + :language: yaml + +You can then load the configuration back to a flow using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_swarm.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config2 + :end-before: .. end-##_Load_Agent_Spec_config2 + + +Next steps +========== + +Now that you have learned how to define a Swarm, you may proceed to :doc:`How to Build Multi-Agent System `. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_swarm.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_template.rst.txt b/26.1.2/_sources/core/howtoguides/howto_template.rst.txt new file mode 100644 index 000000000..7a1ed4961 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_template.rst.txt @@ -0,0 +1,356 @@ +:orphan: + +.. ^ REMOVE THIS AND ADD THE HOW-TO GUIDE TO index.rst file + +================================ +How to {specific task/goal here} +================================ + + +.. Title Guidelines: +.. - Capitalize each word in the main title, except for articles or prepositions +.. - Start with "How to" followed by an action verb +.. - Be specific and action-oriented +.. - Use present tense +.. - Examples of good titles: +.. - How to create stateful tools +.. - How to use a datastore to retrieve data + + +.. admonition:: Prerequisites + + This guide assumes familiarity with the following concepts: + + - {Prerequisite 1 with link to documentation} + - {Prerequisite 2 with link to documentation} + - {Additional prerequisites as needed} + +.. Prerequisites should: +.. - Link to relevant documentation +.. - Be specific and minimal +.. - Only include truly necessary knowledge +.. See at the end of the document for common admonition to use + + +.. warning:: + {Include any version requirements, dependencies, or critical setup information} + +{Brief introduction to the problem and solution} + +This guide will show you how to: + +- {Specific outcome 1} +- {Specific outcome 2} +- {Specific outcome 3} + +.. Describe clearly the problem or task that the guide shows the user how to solve. +.. The overview should: +.. - Be 2-3 paragraphs maximum +.. - Clearly state what problem this guide solves +.. - (when required) Explain why this solution is useful +.. - (when required) Mention any important limitations or considerations + +.. note:: + {Important contextual information or key considerations} + +.. .. To uncomment for actual use +.. .. image:: ../_static/path/to/image.svg +.. :align: center +.. :scale: 70% +.. :alt: {Descriptive alt text for accessibility} + +.. Try to include visual elements whenever possible (e.g. diagrams made with diagram.net) +.. Image Guidelines: +.. - Use SVG format for diagrams (can directly export from draw.io (aka diagrams.net)) +.. - Always include alt text +.. - Use scale between 40-100% +.. - Center align most images + +.. For main content of the how-to guide: +.. - Maximum 2 levels of headings (sections and sub-sections) +.. - Write all subheadings in sentence case +.. - The content can be sequential (e.g. Setup -> Step 1 -> Step 2), +.. or can be an enumeration of different ways to achieve a similar outcome +.. - Keep the section name simple (with as few words as possible) +.. - Each example should be focused and achievable +.. - Encapsulate all function / class / variables with double backticks, e.g. ``Agent`` +.. - Use :doc:`Name ` for linking to different parts of the documentation +.. - Use :ref:`ClassName ` for linking to the API reference +.. - Alternative to :ref: is :meth:`.LlmModelFactory.from_config` +.. or :meth:`~wayflowcore.models.llmmodelfactory.LlmModelFactory.from_config` +.. - Use `text `_ for external links +.. - check https://docutils.sourceforge.io/docs/user/rst/quickref.html for more information + + +.. Note: Choose section name that make sense for the how-to guide. +.. If using verbs, use imperative mood (e.g. Define flow, Add Y, ...) + +Basic implementation +===================== + +.. - Start with the simplest working example +.. - Break into clear, logical steps +.. - Use concrete, runnable code +.. - Explain only what's necessary, refer to conceptual guides otherwise +.. - Link to related documentation frequently + +.. Code blocks can be used, but prefer the use of literalinclude directives + +.. code-block:: python + + {Basic implementation code} + +.. Imports should be done little by little and not everything in the beginning +.. For every new import, add a clickable link to the API reference as follows: + +API Reference: :ref:`LlmModelFactory ` | :ref:`Agent ` + +.. .. To uncomment for actual use +.. .. literalinclude:: ../code_examples/example.py +.. :language: python +.. :start-after: .. start-code_snippet: +.. :end-before: .. end-code_snippet + +.. the same should be done for literal include + +API Reference: :ref:`LlmModelFactory ` | :ref:`Agent ` + +.. note:: + {Important details about the implementation} + +.. - Explain key parts of the code +.. - Use Python code blocks sparingly, prefer the use of literalinclude +.. - Link to the relevant API documentation + + +.. If multiple options exist, use tabs + +.. tabs:: + + .. tab:: Option 1 + + .. code-block:: python + + # code for option 1 + + .. tab:: Option 2 + + .. code-block:: python + + # code for option 2 + +.. with additional references after the tabs + +API Reference: :ref:`LlmModelFactory ` | :ref:`Agent ` + + +.. danger:: + .. Use directives inside the how-to guide to give addition information when needed + + Never commit sensitive information like API keys. Use environment variables: + + .. code-block:: python + + import os + from dotenv import load_dotenv + + load_dotenv() + API_KEY = os.getenv("API_KEY") + + +(OPTIONAL) Alternative Approaches +================================== + +.. Only use when necessary, remove the (Optional) when using +.. - Show different ways to accomplish the same task +.. - Explain when to use each approach +.. - Keep focused on the specific task + +Method 1: {Approach name} +------------------------- + +.. .. To uncomment for actual use +.. .. literalinclude:: ../code_examples/example.py +.. :language: python +.. :start-after: .. start-code_snippet: +.. :end-before: .. end-code_snippet + +.. with additional references + +API Reference: :ref:`LlmModelFactory ` | :ref:`Agent ` + +.. important:: + {Critical information about this approach} + +Method 2: {Alternative approach} +-------------------------------- + +.. .. To uncomment for actual use +.. .. literalinclude:: ../code_examples/example.py +.. :language: python +.. :start-after: .. start-code_snippet: +.. :end-before: .. end-code_snippet + +.. with additional references + +API Reference: :ref:`LlmModelFactory ` | :ref:`Agent ` + +.. tip:: + {When to use one approach over another} + + +(Optional) Advanced Usage +========================== + +.. - Only include if necessary, remove the (Optional) when using +.. - Focus on practical applications +.. - Keep examples concrete + +.. .. To uncomment for actual use +.. .. literalinclude:: ../code_examples/example.py +.. :language: python +.. :start-after: .. start-code_snippet: +.. :end-before: .. end-code_snippet + +.. warning:: + {Important caveats or limitations} + + +(Optional) Common Patterns and Best Practices +============================================== + +.. (Only include when relevant), remove the (Optional) when using +.. This section should: +.. - Highlight recommended approaches +.. - Warn about anti-patterns +.. - Provide real-world usage examples +.. - Link to related patterns in documentation + +Pattern 1: {Pattern Name} +------------------------- + +.. .. To uncomment for actual use +.. .. literalinclude:: ../code_examples/example.py +.. :language: python +.. :start-after: .. start-code_snippet: +.. :end-before: .. end-code_snippet + +.. tip:: + {Why and when to use this pattern} + + +(Optional) Troubleshooting +========================== + +.. (Only include when relevant), remove the (Optional) when using +.. Troubleshooting section should: +.. - Address common issues +.. - Provide clear solutions +.. - Include error messages +.. - Link to relevant documentation + + +{Name of Common Problem no.1} +----------------------------- + +**Symptom** + +.. Add 1-2 sentences to explain when such a problem may occur + +.. code-block:: text + + {Error message or problem description} + +.. Explain briefly if necessary why the problem happens +.. (create a ticket if the error message is not clear enough) + +**Solution** + +.. Explain how to solve the encountered problem and potentially give some +.. hints on how to better do things next time not to encounter it anymore + +.. warning:: + {Important cautions related to the solution} + + +Recap +===== + +.. Mandatory section. The recap should: +.. - Summarize what was learned in 1-2 sentences +.. - Add a full code block for easy user access to executable code + +In this how-to guide, we covered how to [...] + +You can find below the full code to run the example shown in this how-to guide. + +.. .. To uncomment for actual use +.. .. collapse:: full code + +.. .. literalinclude:: ../code_examples/how_to_guide.py +.. :language: python +.. :linenos: +.. :start-after: .. full-code: +.. :end-before: .. end-full-code + +.. Note: In the future the full code collapsible will be replaced by buttons +.. at the top of the guide. In the meantime, simply add the full code that can +.. be copy-pasted by the user and executed. + +Next Steps +========== + +.. Next steps should: +.. - Suggest related topics +.. - Link to advanced guides +.. - Provide resources for further learning (optional) + +Now that you've learned {main topic}, you might want to explore: + +.. .. To uncomment for actual use +.. - :doc:`Related Guide 1 <../path/to/guide>` +.. - :doc:`Related Guide 2 <../path/to/guide>` +.. - :doc:`Advanced Topic <../path/to/advanced>` + +(Optional) Additional Resources +------------------------------- + +- `External Resource 1 `_ +- `External Resource 2 `_ +- `Community Forum `_ + + +.. Quick Reference for RST Directives: +.. - .. warning:: - Important cautionary notes +.. - .. danger:: - Critical warnings +.. - .. tip:: - Helpful suggestions +.. - .. note:: - Additional information +.. - .. important:: - Key points +.. - .. seealso:: - other elements the user might be interested in +.. - .. versionadded:: - New feature annotations +.. - .. deprecated:: - Deprecation notices +.. - .. admonition:: - Custom callouts +.. (e.g. Prerequisites, you should not have to write new ones) + +.. General Writing Style Guidelines: +.. - Use active voice: "Configure the assistant" not "The assistant should be configured" +.. - Present tense: "The function returns" not "The function will return" +.. - Be concise: Avoid unnecessary words +.. - Use consistent terminology +.. - Link liberally to other documentation +.. - Keep paragraphs short (3-4 sentences maximum) + +.. Accessibility Guidelines: +.. - Include alt text for all images +.. - Maintain good color contrast in diagrams, use thin lines sparingly +.. - Provide text alternatives for complex diagrams + +.. Cross-linking Strategy: +.. - Link to prerequisite concepts in introduction +.. - Link to API reference for mentioned classes/functions +.. - Link to related how-to guides in Next Steps +.. - Link to tutorials for broader learning +.. - Link to explanation docs for deep dives into concepts +.. - Use :doc:`Name ` for linking to different parts of the documentation +.. - Use :ref:`ClassName ` for linking to API reference +.. - Use `text `_ for external links diff --git a/26.1.2/_sources/core/howtoguides/howto_tooloutputstreaming.rst.txt b/26.1.2/_sources/core/howtoguides/howto_tooloutputstreaming.rst.txt new file mode 100644 index 000000000..fd152c8dc --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_tooloutputstreaming.rst.txt @@ -0,0 +1,219 @@ +.. _top-tooloutputstreaming: + +=================================== +How to Enable Tool Output Streaming +=================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_tooloutputstreaming.py + :link-alt: Tool Output Streaming how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + - Familiarity with WayFlow Agents and Tools + - Basic understanding of async functions in Python + + +In this guide you will: + +- Create a :ref:`Server Tool ` that streams output chunks; +- Consume tool chunk events with an :ref:`Event Listener `. + + +Tool output streaming for Server Tools +====================================== + +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + + +.. note:: + + **What is tool output streaming:** + Tool output streaming lets a tool produce **intermediate outputs** while it is still running, + instead of waiting until the execution completes to return a single final result. + When streaming is enabled, WayFlow emits chunk events as the tool makes progress (this is emitted as + :ref:`ToolExecutionStreamingChunkReceivedEvent `). + This enables UIs and listeners to display partial results in near real time. + The tool's **final output is the last value produced** (i.e., the completed tool result); + earlier values are treated as streamed chunks emitted during execution. + +You can enable tool output streaming by creating an async generator +(i.e., an async callable yielding items with ``yield`` instead of ``return``). + +When running the async tool callable, yielded items are streamed via the event +:ref:`ToolExecutionStreamingChunkReceivedEvent `. + +The last yielded item is treated as the final tool result **and is not streamed.** + +Here is an example using the :ref:`@tool decorator `: + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Define_streaming_tool + :end-before: .. end-##_Define_streaming_tool + + +You can then define an :ref:`EventListener ` to observe the streamed chunks: + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Define_event_listener + :end-before: .. end-##_Define_event_listener + + +Finally, register a listener before running your Agent/Flow: + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Run_agent_with_stream_listener + :end-before: .. end-##_Run_agent_with_stream_listener + + +Tool output streaming for MCP Tools +=================================== + +You can also enable tool output streaming when using MCP tools by wrapping your server-side async callable +with the :ref:`@mcp_streaming_tool ` decorator. + +.. note:: + + Wrapping the callable allows to automatically handle the streaming by using the progress + notification feature from MCP. This feature only works in async code, so you need to write an + async callable to use the output streaming feature for MCP tools. + +Here is an example using the official MCP SDK: + +.. code-block:: python + + import anyio + from typing import AsyncGenerator + from mcp.server.fastmcp import FastMCP + from wayflowcore.mcp.mcphelpers import mcp_streaming_tool + + server = FastMCP( + name="Example MCP Server", + instructions="A MCP Server.", + ) + + @server.tool(description="Stream intermediate outputs, then yield the final result.") + @mcp_streaming_tool + async def my_streaming_tool(topic: str) -> AsyncGenerator[str, None]: + all_sentences = [f"{topic} part {i}" for i in range(2)] + for i in range(2): + await anyio.sleep(0.2) # simulate work + yield all_sentences[i] + yield ". ".join(all_sentences) + + server.run(transport="streamable-http") + + +When using other MCP libraries (e.g., https://gofastmcp.com/), you need to +provide the context class when using the ``mcp_streaming_tool`` wrapper. + +.. code-block:: python + + from fastmcp import FastMCP, Context + + server = FastMCP( + name="Example MCP Server", + instructions="A MCP Server.", + ) + + async def my_tool() -> AsyncGenerator[str, None]: + contents = [f"This is the sentence N°{i}" for i in range(5)] + for chunk in contents: + yield chunk # streamed chunks + await anyio.sleep(0.2) + + yield ". ".join(contents) # final result + + streaming_tool = mcp_streaming_tool(my_tool, context_cls=Context) + server.tool(description="...")(streaming_tool) + + + +From the client-side, you can consume the MCP tool and observe the streamed chunks +using an event listener as shown above with server tools. + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Run_mcp_streaming_tool + :end-before: .. end-##_Run_mcp_streaming_tool + + +.. seealso:: + + For more information read the :doc:`Guide on using MCP Tools ` + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_tooloutputstreaming.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_tooloutputstreaming.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_tooloutputstreaming.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +Recap +===== + +In this guide, you learned how to: + +- Implement streaming Server Tools using async generators +- Listen to tool chunk events and correlate them with executions +- Adjust the streaming cap (including unlimited) + + +Next steps +========== + +Having learned how to stream tool outputs and consume chunk events, you may now proceed to: + +- :doc:`Build Assistants with Tools ` to design richer tool-enabled agents and flows. +- :doc:`Use the Event System ` to implement custom listeners, tracing, and monitoring. + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_tooloutputstreaming.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_tracing.rst.txt b/26.1.2/_sources/core/howtoguides/howto_tracing.rst.txt new file mode 100644 index 000000000..8a1de2286 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_tracing.rst.txt @@ -0,0 +1,238 @@ +.. _top-tracing: + +================================ +How to Enable Tracing in WayFlow +================================ + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_tracing.py + :link-alt: Tracing how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Using agents ` + + +Tracing is a crucial aspect of any application, allowing developers to monitor and analyze the behavior of their system. +In the context of an agentic framework like WayFlow, tracing allows you to understand the interactions between agents, +tools, and other components. + +In this guide, you will learn how to: + +- Create a :ref:`SpanExporter ` +- Set up tracing in WayFlow +- Save your traces in a file + +What is Tracing? +================ + +Tracing refers to the process of collecting and analyzing data about the execution of a program or system. +This data can include information about function calls, variable assignments, and other events that occur during execution. +By analyzing this data, developers can identify performance bottlenecks, debug issues, and optimize their system for better performance. + +Why is Tracing Important? +========================= + +Tracing is essential for several reasons: + +* **Debugging**: Tracing helps developers identify and diagnose issues in their agents. + By analyzing the trace data, they can pinpoint the exact location and cause of errors. +* **Performance Optimization**: Tracing provides insights into the performance characteristics of an agent, enabling + developers to identify bottlenecks and optimize their architectures for better efficiency. +* **Monitoring**: Tracing allows developers to monitor the behavior of their agents in real-time, enabling them + to detect anomalies and respond promptly to issues. + + +Basic implementation +==================== + +To set up tracing in WayFlow, you need to provide an implementation of the :ref:`SpanProcessor ` +and :ref:`SpanExporter ` classes. + +A :ref:`SpanProcessor ` is a common concept in the observability world. +It is a component in the tracing pipeline responsible for receiving and processing spans as they are +created and completed by the application. +:ref:`SpanProcessor ` sit between the tracing backend and the exporter, allowing developers +to implement logic such as batching, filtering, modification, or immediate export of :ref:`Spans `. +When a :ref:`Span ` ends, the :ref:`SpanProcessor ` determines what happens to it next, +whether it’s sent off immediately, or collected for more efficient periodic export (e.g., doing batching). +This flexible mechanism enables customization of trace data handling before it's ultimately exported to backend observability systems. + +A :ref:`SpanExporter ` is a component that is responsible for sending finished spans, along with their +collected trace data, from the application to an external backend or observability system for storage and analysis. +The exporter receives spans from the :ref:`SpanProcessor ` and translates them into the appropriate format +for the target system, such as `LangFuse `_, +`LangSmith `_, or +`OCI APM `_. +Exporters encapsulate the logic required to connect, serialize, and transmit data, allowing OpenTelemetry +to support a wide range of backends through a consistent, pluggable interface. +This mechanism enables seamless integration of collected trace data with various monitoring and tracing platforms. + +In the following sections you will learn how to implement a combination of SpanProcessor and SpanExporter +that can export traces to a file. + +SpanProcessor and SpanExporter +------------------------------ + +.. danger:: + Several security concerns arise when implementing SpanProcessors and SpanExporters, which include, + but they are not limited to, the security of the network used to export traces, and the sensitivity of the + information exported. Please refer to our :doc:`Security Guidelines <../security>` for more information. + +As partially anticipated in the previous section, the most simple implementation of a :ref:`SpanProcessor ` +is the one that exports the received :ref:`Span ` as-is, without any modification, as soon as the :ref:`Span ` is closed. +This implementation is provided by ``wayflowcore``, and it is called :ref:`SimpleSpanProcessor `. +You will use an instance of this :ref:`SpanProcessor ` in this guide. + +For what concerns the :ref:`SpanExporter `, you can implement a version of it that just prints +the information contained in the :ref:`Spans ` to a file at a given path. +The implementation can focus on the `export` method, that opens the file in `append` mode, and it prints in it +the content of the :ref:`Spans ` retrieved through the `to_tracing_info` method. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Span_Exporter_Setup + :end-before: .. end-##_Span_Exporter_Setup + +You can now combine :ref:`SimpleSpanProcessor ` with the `FileSpanExporter` you just implemented +to set up the basic components that will let you export traces to the desired file. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Tracing_Basics + :end-before: .. end-##_Tracing_Basics + +Tracking an agent +----------------- + +Now that you have everything you need to process and export traces, you can work on your agent. + +In this example, you are going to build a simple calculator agent with four tools, one for each of the basic operations: +addition, subtraction, multiplication, division. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Build_Calculator_Agent + :end-before: .. end-##_Build_Calculator_Agent + +We now run your agent enabling traces, and using the ``FileSpanExporter`` in order to export the traces in a file. +To do that, just wrap the execution loop of our agent in a :ref:`Trace ` context manager. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Agent_Execution_With_Tracing + :end-before: .. end-##_Agent_Execution_With_Tracing + +You can now run our code and inspect the traces saved in your file. + +Emitting Agent Spec Traces +========================== + +Open Agent Specification Tracing (short: Agent Spec Tracing) is an extension of +Agent Spec that standardizes how agent and flow executions emit traces. +It defines a unified, implementation-agnostic semantic for, Events, Spans, Traces, and SpanProcessors, with +the same semantic presented for WayFlow in this guide. + +WayFlow offers an ``EventListener`` called :ref:`AgentSpecEventListener ` that +makes WayFlow components emit traces according to the Agent Spec Tracing standard. +Here's an example of how to use it in your code. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Enable_Agent_Spec_Tracing + :end-before: .. end-##_Enable_Agent_Spec_Tracing + + +Agent Spec Exporting/Loading +============================ + +You can export the agent configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_tracing.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_tracing.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_tracing.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginPromptTemplate`` + - ``PluginRemoveEmptyNonUserMessageTransform`` + - ``ExtendedAgent`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + +Using OpenTelemetry SpanProcessors +================================== + +`OpenTelemetry `_ is an open-source observability framework that provides standardized APIs and +libraries to collect, process, and export telemetry data from distributed systems. +This standard is agnostic with respect to the domain of application, so it can be easily +adopted also for tracing in agentic frameworks. + +Tracing in WayFlow is largely inspired by the OpenTelemetry standard, therefore most of the +concepts and APIs overlap. +For this reason, ``wayflowcore`` offers the implementation of two ``SpanProcessors`` that follow +the OpenTelemetry standard: + +- :ref:`OtelSimpleSpanProcessor `: A span processor that exports spans one by one +- :ref:`OtelBatchSpanProcessor `: A span processor that exports spans in batches + +These span processors wrap the OpenTelemetry implementation, transform WayFlow spans into OpenTelemetry ones, +and emulate the expected behavior of the processor. +Moreover, they allow using OpenTelemetry compatible ``SpanExporter``, like, for example, +those offered by the `OpenTelemetry Exporters library `_. + +Next steps +========== + +Now that you've learned tracing in WayFlow, you might want to apply it in other scenarios: + +- :doc:`How to Build a Swarm of Agents ` +- :doc:`How to Build Multi-Agent Assistants ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_tracing.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_userconfirmation.rst.txt b/26.1.2/_sources/core/howtoguides/howto_userconfirmation.rst.txt new file mode 100644 index 000000000..af1e03da8 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_userconfirmation.rst.txt @@ -0,0 +1,126 @@ +.. _top-userconfirmation: + +================================================== +How to Add User Confirmation to Tool Call Requests +================================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_userconfirmation.py + :link-alt: User Confirmation how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Agents <../tutorials/basic_agent>` + - :doc:`Building Assistants with Tools ` + +WayFlow :ref:`Agents ` can be equipped with :doc:`Tools <../api/tools>` to enhance their capabilities. +However, end users may want to confirm or deny tool call requests emitted from the agent. + +This guide shows you how to achieve this with the :ref:`ServerTool `. You can also do this using a :ref:`ClientTool ` + + +Basic implementation +==================== + +In this example, you will build a simple :ref:`Agent ` equipped with three tools: + +* A tool to add numbers +* A tool to subtract numbers +* A tool to multiply numbers + +This guide requires the use of an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +To learn more about the different LLM providers, read the guide on :doc:`How to Use LLMs from Different Providers `. +The sample LLM used for this guide is defined as follows: + +.. literalinclude:: ../code_examples/howto_userconfirmation.py + :language: python + :start-after: .. start-##Configure_LLM + :end-before: .. end-##Configure_LLM + + +Creating the tools +------------------ + +Sometimes you will want to ask for user confirmation before executing certain tools. You can do this by using a :ref:`ServerTool ` with the flag ``requires_confirmation`` set to ``True``. This will raise a ``ToolExecutionConfirmationStatus`` +whenever the ``Agent`` tries to execute the tool. We set the multiply_numbers tool to not require confirmation to highlight the differences. Note that the ``requires_confirmation`` flag can be used for any WayFlow tool. + +.. literalinclude:: ../code_examples/howto_userconfirmation.py + :language: python + :start-after: .. start-##Create_tools + :end-before: .. end-##Create_tools + + +Creating the user-side execution logic +---------------------------------------- + +To enable users to accept or deny tool call requests, you add simple validation logic before executing the tools requested by the agent. + +.. literalinclude:: ../code_examples/howto_userconfirmation.py + :language: python + :start-after: .. start-##Create_tool_execution + :end-before: .. end-##Create_tool_execution + +Here, you simply loop until the user answers whether to accept the tool request or reject it. You can accept the tool request by using the ``status.confirm_tool_execution`` method. +While accepting, you need to specify the specific tool request and you also have the option to add ``modified_args`` in this method to change the arguments of the called tool. +Similarly, for rejection you can use the ``status.reject_tool_execution`` with an optional ``reason`` so that the ``Agent`` can take the reason into account while planning the next action to take. + + +Creating the agent +------------------ + +Finally, you create a simple ``Agent`` to test the execution code written in the previous section. + +.. literalinclude:: ../code_examples/howto_userconfirmation.py + :language: python + :start-after: .. start-##Create_agent + :end-before: .. end-##Create_agent + + +Running the agent in an execution loop +-------------------------------------- + +Now, you create a simple execution loop to test the agent. +In this loop, you can input the instructions you want the agent to execute and test it out for yourself! + +.. literalinclude:: ../code_examples/howto_userconfirmation.py + :language: python + :start-after: .. start-##Run_tool_loop + :end-before: .. end-##Run_tool_loop + +Recap +===== + +In this guide, you learned how to support user-side confirmation for tool call requests by using ``ServerTool``. + + +Next steps +========== + +Having learned how to add user confirmation for tool calls, you may now proceed to: + +- :doc:`How to Create Conditional Transitions in Flows ` +- :doc:`How to Create a ServerTool from a Flow ` + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_userconfirmation.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_userinputinflows.rst.txt b/26.1.2/_sources/core/howtoguides/howto_userinputinflows.rst.txt new file mode 100644 index 000000000..d5cc815ff --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_userinputinflows.rst.txt @@ -0,0 +1,181 @@ +.. _top-userinputinflows: + +================================== +How to Ask for User Input in Flows +================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_userinputinflows.py + :link-alt: User input in flows how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with: + + - :doc:`Flows <../tutorials/basic_flow>` + - :doc:`Tools <../api/tools>` + +WayFlow allows you to build powerful automation and agentic workflows. +In many real-world scenarios, your flows will need to request and incorporate input from a human user — either to execute a particular action, validate a decision, or simply continue the process. + +This guide explains how to design flows that pause for user input, receive responses, and resume execution seamlessly (also known as Human-in-the-loop (HITL) machine learning). + +Overview +======== + +There are two standard patterns for requesting user input within a flow: + +- **Simple user requests** (e.g., prompting the user for a question or parameter) +- **Interactive/branching patterns** (e.g., asking for confirmation before performing an action, with logic based on the user's response) + +This guide will show you how to: + +- Add a user input request to your flow using the :ref:`InputMessageStep ` +- Connect user responses to further steps for flexible flow logic +- Chain multiple interactions, including branching for confirmation scenarios + +.. note:: + User input is always delivered via ``InputMessageStep``. WayFlow's status objects make it easy to detect when input is required and to resume execution once the user has responded. + + +Basic implementation +==================== + +This guide requires the use of an LLM. +WayFlow supports several LLM API providers. +Select an LLM from the options below: + +.. include:: ../_components/llm_config_tabs.rst + +The easiest way to capture user input is with ``InputMessageStep``. +Used in combination with an execution loop, this step is used to prompt the user for input, +pause flow execution, and deliver the user's response into the flow's data context for +use by subsequent steps. + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Create_Simple_Flow + :end-before: .. end-##_Create_Simple_Flow + +API Reference: :ref:`Flow ` | :ref:`InputMessageStep ` | :ref:`CompleteStep ` + +You can then execute this flow as shown below. Notice how the execution is paused until the user enters their input: + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Execute_Simple_Flow + :end-before: .. end-##_Execute_Simple_Flow + +.. note:: + When ``conversation.execute()`` returns a ``UserMessageRequestStatus``, you must append a user message (with ``conversation.append_user_message(...)``) to continue the flow. + + +Advanced pattern: Request user input for tool calls or approvals +================================================================ + +.. seealso:: + + For details on enabling client-side confirmations, see the guide :doc:`How to Add User Confirmation to Tool Call Requests `. + +In some cases, it is necessary not only to collect a user's initial input but also request confirmation before executing certain actions — such as validating tool calls or branching flow execution based on user responses. + +The following example demonstrates a more sophisticated flow. +The flow pauses both for the user's main request and again for tool call confirmation, using branching to repeat or skip steps depending on the response. + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Create_Complex_Flow + :end-before: .. end-##_Create_Complex_Flow + +API Reference: :ref:`Flow ` | :ref:`InputMessageStep ` | :ref:`BranchingStep ` | :ref:`ToolExecutionStep ` + +.. note:: + The ``InputMessageStep`` can be reused at multiple points in a flow for different types of input—questions, approvals, parameter selection, etc. Use ``BranchingStep`` to control your logic flow depending on the user's reply. + +To drive this advanced flow, you execute and interact with the agent as follows: + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Execute_Complex_Flow + :end-before: .. end-##_Execute_Complex_Flow + +.. tip:: + Design your ``message_template`` and branching mapping to ensure robust, user-friendly interactions. + You can combine user input at any point with decision logic for flexible, agent-like flows. + + You can also use :ref:`CatchExceptionStep ` to handle issues such as user typing something else than "y/n". + +Agent Spec Exporting/Loading +============================ + +You can export the flow configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Export_Config_to_Agent_Spec + :end-before: .. end-##_Export_Config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_userinputinflows.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_userinputinflows.yaml + :language: yaml + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + + +.. literalinclude:: ../code_examples/howto_userinputinflows.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_Config + :end-before: .. end-##_Load_Agent_Spec_Config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginInputMessageNode`` + - ``PluginConstantContextProvider`` + - ``PluginExtractNode`` + - ``ExtendedToolNode`` + - ``ExtendedFlow`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +In this guide, you learned how to request and handle user input and approval inside WayFlow flows using ``InputMessageStep``, as well as how to combine input with branching for selective actions. You may now proceed to: + +- :doc:`How to Create Conditional Transitions in Flows ` +- :doc:`How to Add User Confirmation to Tool Call Requests ` +- :doc:`How to Create a ServerTool from a Flow ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_userinputinflows.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/howto_variable.rst.txt b/26.1.2/_sources/core/howtoguides/howto_variable.rst.txt new file mode 100644 index 000000000..7cfb61f44 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/howto_variable.rst.txt @@ -0,0 +1,167 @@ +.. _top-howtovariable: + +============================================== +How to Use Variables for Shared State in Flows +============================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_variable.py + :link-alt: Variable how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>` + +When building Flows, you may need a way to preserve information as it moves from one step to another. +WayFlow provides this through :ref:`Variable `, which serves as the flow’s state — a place where values can be stored, accessed, and updated throughout execution. + +Why to use Variable? + +- **Shared state**: Holds data that multiple steps can share. +- **Intermediate results**: Store partial results and reuse them later. +- **Simpler data flow**: Avoid passing outputs between every step, persist them as state using Variable. + +This guide will show you how to: + +- Define a :ref:`Variable ` in a Flow. +- Read and write its value with :ref:`VariableStep `. + +In this guide, you will see a simple example including defining a ``Variable`` that stores a list of user feedback, +using ``VariableStep`` to insert new feedback into the list and read all collected feedback. + +Define a Variable +================= + +To define a variable, you need to define it with a name, type and optionally, a default value. +The type of variable determines the kind of data it can hold and is defined using ``Property``. +In this case, our ``feedback_variable`` has the type of ``ListProperty`` and the ``item_type`` of ``StringProperty``. + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :linenos: + :start-after: .. start-##_Define_a_Variable + :end-before: .. end-##_Define_a_Variable + +**API Reference:** :ref:`Variable ` | :ref:`Property ` + +Define Flow Steps +================= + +We will define a simple flow including the following steps. + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :linenos: + :start-after: .. start-##_Define_Flow_Steps + :end-before: .. end-##_Define_Flow_Steps + +For simplicity, we pass initial feedback to the ``start_step``, which then routes values to ``write_feedback_1`` and ``write_feedback_2``. +In practice, those inputs could come from other steps (e.g. :ref:`ToolExecutionStep`). + +When updating some variables, the :ref:`VariableStep ` requires the ``write_variables`` that it writes to. It also accepts the following options of write operations: + +- ``VariableWriteOperation.OVERWRITE`` (or ``'overwrite'``) works on any type of variable to replace its value with the incoming value. +- ``VariableWriteOperation.MERGE`` (or ``'merge'``) updates a ``Variable`` of type dict (resp. list), +- ``VariableWriteOperation.INSERT`` (or ``'insert'``) operation can be used to append a single element at the end of a list. + +Here, we choose ``insert`` as we want to append new user feedback to the our list. + +The ``VariableStep`` can also read one or more variables, via the ``read_variables`` parameter. + +The ``VariableStep`` can perform both writes and reads of multiple variables at the same time. +If you configure a step to both write and read a given variable, the value will be written first, and the updated variable will be used for the read operation. + +In general, the input and output descriptors of the ``VariableStep`` correspond to the properties referenced by the variables being written and/or read in this step, respectively. +For example, if the step is configured to overwrite a ``Variable(name="manager", type=DictProperty())``, to extend a ``Variable(name="employees", type=ListProperty(item_type=DictProperty("employee"))``, and to read back the updated value of the employee list, the step would have the following descriptors: + +* An input descriptor ``DictProperty("manager")``, to overwrite the full manager variable; +* An input descriptor ``DictProperty("employee")``, since the extend operation expects a single item of the employees list; +* An output descriptor ``ListProperty("employees", item_type=DictProperty("employee"))`` + +Define a Flow with Variable +=========================== + +Now we connect everything into a flow: two write steps add feedback, and a read step collects it all for output. + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :linenos: + :start-after: .. start-##_Define_a_Flow_with_variable + :end-before: .. end-##_Define_a_Flow_with_variable + +Remember to include your defined variables in the Flow’s ``variables`` parameter. + +Execute the Flow +================ + +Finally, run the flow: + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :linenos: + :start-after: .. start-##_Execute_flow + :end-before: .. end-##_Execute_flow + +Agent Spec Exporting/Loading +============================ + +You can export the assistant configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the AgentSpec assistant configuration. + + .. tabs:: + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_variable.yaml + :language: yaml + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_variable.json + :language: json + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_variable.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginReadVariableNode`` + - ``PluginWriteVariableNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + +Next steps +========== + +Now that you have learned how to use Variables in Flows, you may proceed to :ref:`FlowContextProvider` to learn how to provide context for flow execution. + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_variable.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/index.rst.txt b/26.1.2/_sources/core/howtoguides/index.rst.txt new file mode 100644 index 000000000..234da7699 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/index.rst.txt @@ -0,0 +1,135 @@ +.. _how-to_guides: + +:html_theme.sidebar_secondary.remove: + +How-to Guides +============= + +Within this section, you will find answers to "How do I..." types of questions. +The proposed guides are goal-oriented and concrete, as they are meant to help you complete a specific task. +Each code example in these how-to guides is self-contained and can be executed with :doc:`WayFlow <../index>`. + +For comprehensive, end-to-end walkthroughs, refer to the :doc:`Tutorials <../tutorials/index>`. +For detailed descriptions of every class and function, see the :doc:`API Reference <../api/index>`. + + +Building Assistants +------------------- + +WayFlow offers a wide range of features for building :ref:`Agents `, :ref:`Flows ` as well +as multi-agent patterns such as :doc:`hierarchical multi-agent ` and :ref:`Swarm `. +These how-to guides demonstrate how to use the main features to create and customize your assistants. + +.. toctree:: + :maxdepth: 1 + + Change Input and Output Descriptors of Components + Use Asynchronous APIs + +.. toctree:: + :maxdepth: 1 + :caption: LLMs + + Install and Use Ollama + Specify the Generation Configuration when Using LLMs + Configure Retries on LLMs and Remote Components + Use LLM from Different LLM Sources and Providers + Handle long context with agents + +.. toctree:: + :maxdepth: 1 + :caption: Agents + + Create a ReAct Agent + How to Send Images to LLMs and Agents + Use OCI Generative AI Agents + How to Connect to A2A Agents + Use Templates for Advanced Prompting Techniques + +.. toctree:: + :maxdepth: 1 + :caption: Flows + + Ask for User Input in Flows + Create Conditional Transitions in Flows + Do Structured LLM Generation in Flows + Add User Confirmation to a Tool Call Request + Catch Exceptions in Flows + Do Map and Reduce Operations in Flows + Run Multiple Flows in Parallel + Build Flows with the Flow Builder + +.. toctree:: + :maxdepth: 1 + :caption: Multi-Agent Patterns + + Use Agents in Flows + Build a Hierarchical Multi-Agent System + Build a Swarm of Agents + Build a ManagerWorkers of Agents + + +.. toctree:: + :maxdepth: 1 + :caption: Deployment + + Serve Agents with WayFlow + Serve Assistants with A2A protocol + +Tools in Assistants +------------------- + +Equipping AI assistants with tools unlock key capabilities such as being able to fetch data, +take action, and connect to external data and systems. These guides cover how to leverage the +several tools features available in WayFlow, such as :ref:`server-side tools `, +:ref:`client-side tools `, tools to perform :ref:`remote API calls `, +and support for :ref:`Model Context Protocol (MCP) tools `. + +.. toctree:: + :maxdepth: 1 + + Build Assistants with Tools + Create Tools with Multiple Outputs + Convert Flows to Tools + Connect MCP tools to Assistants + Enable Tool Output Streaming + Do Remote API Calls with Tokens + + +Configuration and State Management +---------------------------------- + +These guides demonstrate how to configure the components of assistants built with WayFlow. + +.. toctree:: + :maxdepth: 1 + + Load and Execute an Agent Spec Configuration + Serialize and Deserialize Flows and Agents + Serialize and Deserialize Conversations + Build a New WayFlow Component + Enable Tracing + Use the Event System + + +Data in Assistants +------------------ + +.. toctree:: + :maxdepth: 1 + + Connect Assistants to Your Data + Use Embedding Models from Different Providers + Use Variables for Shared State in Flows + Synthesize Data in WayFlow + Build RAG-Powered Assistants + + +Assistant Testing and Evaluation +-------------------------------- + +.. toctree:: + :maxdepth: 1 + + Evaluate Assistants + Evaluate Conversations diff --git a/26.1.2/_sources/core/howtoguides/installing_ollama.rst.txt b/26.1.2/_sources/core/howtoguides/installing_ollama.rst.txt new file mode 100644 index 000000000..2f5ca6487 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/installing_ollama.rst.txt @@ -0,0 +1,135 @@ +============================= +How to Install and Use Ollama +============================= + +.. admonition:: Prerequisites + + This guide does not assume any prior knowledge about WayFlow. + + +`Ollama `_ is a lightweight framework for running Large Language Models (LLMs) on a local machine. +Ollama does not require an internet connection to run the models, +which makes it a great solution to quickly prototype WayFlow assistants without relying on external LLM providers. + +This how-to guide demonstrates how to install and use Ollama to build AI-powered assistants with WayFlow. + + +Installing Ollama +================= + +This section shows how to install Ollama on macOS. + +**Step 1:** Go to the `Ollama `_ website and click on ``Download``. + +.. image:: ../_static/ollama/ollama_frontpage.png + :align: center + :width: 80% + +\ + +**Step 2:** Navigate to your Downloads folder and extract the `ollama`` archive. On macOS, double-click the _.zip_ file to unzip it. + +.. image:: ../_static/ollama/ollama_extract1_2.png + :align: center + :width: 80% + +\ + +**Step3:** Double-click the Ollama application to launch it. When prompted, click Move to Applications to relocate it to your Applications folder. +Then, follow the on-screen instructions to complete the installation. + +.. image:: ../_static/ollama/ollama_extract2_2.png + :align: center + :width: 80% + +\ + +You may be prompted to enter your password to complete the installation. + +.. image:: ../_static/ollama/ollama_install1_3.png + :width: 30% + +.. image:: ../_static/ollama/ollama_install2_3.png + :width: 30% + +.. image:: ../_static/ollama/ollama_install3_3.png + :width: 30% + +\ + + +Testing Ollama in the terminal +============================== + +To try Ollama in the terminal, open a terminal window and run ``ollama run llama3.2`` to use the latest model. + +.. image:: ../_static/ollama/ollama_in_terminal.png + :align: center + :width: 80% + +\ + +To exit the conversation, simply enter ``/bye``. + +To display the list of running models, use the command ``ollama ps``. + +.. warning:: + Exiting the terminal is not sufficient to stop the model. To stop the model, use the command ``ollama stop ``. + + +Using Ollama in WayFlow +=========================== + +To use Ollama within WayFlow, set the model type to `ollama`` and specify the desired model ID (for example, ``llama3.2``). + +**Important:** Before using a model in WayFlow, ensure it is downloaded and running. Execute ``ollama run `` in your terminal to start the model. +For a comprehensive list of available models, visit the `Ollama Library `_. + +.. note:: + Ollama is designed to run lightweight models, which typically include smaller models (e.g., Llama3.2 with 1B or 3B parameters) or heavily quantized models (e.g., Llama3.1 with 4-bit quantization). + As a consequence, their performance may not match the performance of the original models. + +.. code:: python + + from wayflowcore.agent import Agent + from wayflowcore.models import OllamaModel + + llm = OllamaModel( + model_id="llama3.2:1b", # Update this with the model you will use. + ) + + llm = LlmModelFactory.from_config(OLLAMA_CONFIG) + assistant = Agent(llm=llm) + + conversation = assistant.start_conversation() + conversation.append_user_message("I need help regarding my sql query") + conversation.execute() + + # get the assistant's response to your query + assistant_answer = conversation.get_last_message().content + # I'd be happy to help with your SQL query... + + print(assistant_answer) + + +.. note:: + By default Ollama binds port 11434. You can specify a custom host port by specifying the ``"host_port"`` key in the model configuration. For example: + + .. code:: python + + llm = OllamaModel( + model_id="llama3.2:1b", # Update this with the model you will use. + host_port: "localhost:11435", + ) + + +After using an Ollama-hosted model, it is advisable to stop the model to free up system resources. +To do this, first list all currently running model names by executing ``ollama ps``, and then run ``ollama stop `` to stop the desired model. + + +Next steps +========== + +In this guide, you have learned how to install and use Ollama to start building assistants with WayFlow. + +To learn more, check out tutorials on how :doc:`Build a Simple Conversational Assistant with Agents <../tutorials/basic_agent>` as well as how to :doc:`Build a Simple Fixed-Flow Assistant with Flows <../tutorials/basic_flow>`. diff --git a/26.1.2/_sources/core/howtoguides/io_descriptors.rst.txt b/26.1.2/_sources/core/howtoguides/io_descriptors.rst.txt new file mode 100644 index 000000000..25924c486 --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/io_descriptors.rst.txt @@ -0,0 +1,169 @@ +.. _top-howiodescriptors: + +======================================================== +How to Change Input and Output Descriptors of Components +======================================================== + +.. |python-icon| image:: ../../_static/icons/python-icon.svg + :width: 40px + :height: 40px + +.. grid:: 2 + + .. grid-item-card:: |python-icon| Download Python Script + :link: ../end_to_end_code_examples/howto_io_descriptors.py + :link-alt: IO descriptors how-to script + + Python script/notebook for this guide. + +.. admonition:: Prerequisites + + This guide assumes familiarity with :doc:`Flows <../tutorials/basic_flow>`. + +WayFlow components such as :ref:`Agents `, :ref:`Flows `, and :ref:`Steps ` accept inputs and produce outputs. +These inputs and outputs allow you to pass values to be used, and return some new values. +You can inspect the input/output descriptors on classes that inherit from ``ComponentWithInputsOutputs`` by accessing the ``input_descriptors`` and ``output_descriptors`` attributes, respectively. +See the :ref:`Property ` API documentation to learn more about the IO typing system operations. + +Sometimes, it is helpful to change their description, either because the type is not specific enough, or if you want +to specify a default value. + +This guide will show you how to **override the default input and output descriptions of Agents, Flows, or Steps**. + +Basic implementation +==================== + +When creating a step, input and output descriptors are automatically detected based on the step's configuration. + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Auto_IO_detection + :end-before: .. end-##_Auto_IO_detection + +In this case, the input descriptor ``service`` does not have a default value, and the description is not very informative. +To improve the user experience, you can provide a more informative description and set a default value by overriding the input descriptors: + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Specify_input_descriptor + :end-before: .. end-##_Specify_input_descriptor + +.. note:: + + Since a step requires specific variables to work well, the overriding descriptor must have the same ``name`` as the original descriptor. + +The same process can be applied to output descriptors. + +Refining a type +=============== + +In certain situations, the automatic detection of input and output types may not determine the appropriate type for a variable. +For example, consider the following step where an ``AnyProperty`` input is detected: + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Default_any_descriptor + :end-before: .. end-##_Default_any_descriptor + +Here, the service input is expected to be a list. +To improve clarity, you can override the ``AnyProperty`` descriptor to specify the expected type: + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_List_descriptor + :end-before: .. end-##_List_descriptor + +.. note:: + + Currently, type validation is not implemented. When overriding a descriptor's type, make sure to specify the correct type to prevent runtime crashes during step execution. + + +Changing the name of a descriptor +================================= + +Sometimes, the default name of an input or output descriptor can be complex or unclear. + +In this case, you can not just modify the names of the ``input_descriptors`` or ``output_descriptors``, as these names are integral to mapping between new and default descriptors. + +You can still rename the input or output descriptor of a ``Step`` by using ``input_mapping`` or ``output_mapping``. +These mappings associate the default descriptor names (keys) with the desired new names (values). +The associated ``input_descriptors`` and ``output_descriptors`` need to reflect these new names accordingly. + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Rename_a_descriptor + :end-before: .. end-##_Rename_a_descriptor + +Without providing the ``input_mapping`` value, the step will not recognize the input descriptor name and will raise an error. + +.. code-block:: python + + step = InputMessageStep( + message_template="Hi {{unclear_var_name}}. How are you doing?", + input_descriptors=[StringProperty(name="username", description="name of the current user")], + ) + # ValueError: Unknown input descriptor specified: StringProperty(name='username', description='name of the current user'). Make sure there is no misspelling. + # Expected input descriptors are: [StringProperty(name='unclear_var_name', description='"unclear_var_name" input variable for the template')] + + +Agent Spec Exporting/Loading +============================ + +You can export the step configuration to its Agent Spec configuration using the ``AgentSpecExporter``. + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Export_config_to_Agent_Spec + :end-before: .. end-##_Export_config_to_Agent_Spec + + +Here is what the **Agent Spec representation will look like ↓** + +.. collapse:: Click here to see the assistant configuration. + + .. tabs:: + + .. tab:: JSON + + .. literalinclude:: ../config_examples/howto_io_descriptors.json + :language: json + + .. tab:: YAML + + .. literalinclude:: ../config_examples/howto_io_descriptors.yaml + :language: yaml + + +You can then load the configuration back to an assistant using the ``AgentSpecLoader``. + +.. literalinclude:: ../code_examples/howto_io_descriptors.py + :language: python + :start-after: .. start-##_Load_Agent_Spec_config + :end-before: .. end-##_Load_Agent_Spec_config + +.. note:: + + This guide uses the following extension/plugin Agent Spec components: + + - ``PluginInputMessageNode`` + + See the list of available Agent Spec extension/plugin components in the :doc:`API Reference <../api/agentspec>` + + +Next steps +========== + +Having learned how to override the default input and output descriptions of a component, you may now proceed to: + +- :doc:`How to Do Structured LLM Generation in Flows ` +- :doc:`How to Use Agents in Flows ` + + +Full code +========= + +Click on the card at the :ref:`top of this page ` to download the full code for this guide or copy the code below. + +.. literalinclude:: ../end_to_end_code_examples/howto_io_descriptors.py + :language: python + :linenos: diff --git a/26.1.2/_sources/core/howtoguides/llm_from_different_providers.rst.txt b/26.1.2/_sources/core/howtoguides/llm_from_different_providers.rst.txt new file mode 100644 index 000000000..3f013a6fb --- /dev/null +++ b/26.1.2/_sources/core/howtoguides/llm_from_different_providers.rst.txt @@ -0,0 +1,405 @@ +============================================ +How to Use LLMs from Different LLM Providers +============================================ + +WayFlow supports several LLM API providers. The available LLMs are: + +- :ref:`OpenAIModel ` +- :ref:`GeminiModel ` +- :ref:`OpenAICompatibleModel ` +- :ref:`OCIGenAIModel ` +- :ref:`VllmModel ` +- :ref:`OllamaModel ` + +Their configuration is specified directly to their respective class constructor. +This guide will show you how to configure LLMs from different LLM providers with examples and notes on usage. + +Basic implementation +-------------------- + +Currently, defining a configuration dictionary and passing it to the :meth:`.LlmModelFactory.from_config` method is a convenient way to instantiate a particular LLM model in WayFlow. +However, you can also achieve this by directly instantiating the model classes, providing flexibility for more customized setups. + +You can find a detailed description of each supported model type in this guide, demonstrating both methods — using the configuration dictionary and direct instantiation — for each model. + +.. _openaicompatible_custom_tls: + +Custom TLS Certificates for OpenAI-Compatible Endpoints +------------------------------------------------------- + +``OpenAICompatibleModel``, ``VllmModel`` and ``OllamaModel`` accept optional TLS certificate parameters for HTTPS endpoints that are not covered by the system CA store. + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. openai-compatible-tls-start + :end-before: .. openai-compatible-tls-end + +Use ``ca_file`` to trust a custom certificate authority, and use ``key_file`` together with ``cert_file`` for mutual TLS (mTLS). +These certificate fields are runtime-only and are intentionally omitted from WayFlow serialization output. + +OCI GenAI Model +--------------- + +`OCI GenAI Model `_ is powered by `OCI Generative AI `_. + +**Parameters** + +.. option:: model_id: str + + Name of the model to use. A list of the available models is given in + `Oracle OCI Documentation `_ + under the Model Retirement Dates (On-Demand Mode) section. + +.. option:: generation_config: dict, optional + + Default parameters for text generation with this model. + Example: + + .. code-block:: python + + generation_config = LlmGenerationConfig(max_tokens=256, temperature=0.8, top_p=0.95) + +.. option:: client_config: OCIClientConfig, optional + + OCI client config to authenticate the OCI service. + See the below examples and :ref:`ociclientconfigclassesforauthentication` for the usage and more information. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. oci-start + :end-before: .. oci-end + +.. collapse:: Equivalent code example utilizing the LlmModelFactory class. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. oci-llmfactory-start + :end-before: .. oci-llmfactory-end + +.. collapse:: Equivalent code example utilizing the OCIUserAuthenticationConfig class (API_KEY authentication without config/key files). + + WayFlow allows users to authenticate OCI GenAI service using a user API key without relying on a local config file and a key file. + + Instead of using a config file, the values of config parameters can be specified in the :ref:`OCIUserAuthenticationConfig `. + + .. literalinclude:: ../code_examples/example_ociuserauthentication.py + :language: python + :start-after: .. start-userauthenticationconfig: + :end-before: .. end-userauthenticationconfig + + .. note:: + + The user authentication config parameters are sensitive information. This information will not be included when serializing a flow (there will be just an empty dictionary instead). + + You can create a client configuration with the user authentication configuration. + + .. literalinclude:: ../code_examples/example_ociuserauthentication.py + :language: python + :start-after: .. start-clientconfig: + :end-before: .. end-clientconfig + + + Then create an ``OCIGenAIModel`` object: + + .. literalinclude:: ../code_examples/example_ociuserauthentication.py + :language: python + :start-after: .. start-ocigenaimodel: + :end-before: .. end-ocigenaimodel + + Alternatively, you can use the :meth:`.LlmModelFactory.from_config` to create an ``OCIGenAIModel`` object: + + .. literalinclude:: ../code_examples/example_ociuserauthentication.py + :language: python + :start-after: .. start-llmmodelfactory: + :end-before: .. end-llmmodelfactory + +**Notes** + +- Make sure to properly set up authentication configuration. +- Make sure that you have the ``oci>=2.134.0`` package installed. With your WayFlow environment activated, you can install the package as follows: + + .. code-block:: bash + + pip install oci>=2.134.0 + +.. note:: + We recommend to encapsulate your code with ``if __name__ == "__main__":`` to avoid any unexpected issues. + +.. important:: + If, when using the ``INSTANCE_PRINCIPAL``, the response of the model returns a ``404`` error, + check if your instance is listed in the dynamic group and has the right privileges. + Otherwise, ask someone with administrative privileges to grant your OCI Compute instance the ability to authenticate as an Instance Principal. + You need to have a Dynamic Group that includes the instance and a policy that allows this dynamic group to manage OCI GenAI services. + +.. _subsection-api-key-gen: + +Using the API_KEY authentication method +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to use the ``API_KEY`` authentication method, generating and setting a new ``.pem`` OCI key is necessary. +The following steps will guide you through the generation and setup process: + + 1. Login to the OCI console. + 2. In the navigation bar, select the **Profile** menu and then navigate to **User settings** or **My profile**, depending on the option that you see. + 3. Under **Resources**, select **API Keys**, and then select **Add API Key**. + 4. Select **Generate API Key Pair** in the Add API Key dialog. + 5. Select **Download Private Key** and save the private key file (the *.pem* file) in the *~/.oci/config* directory. (If the *~/.oci/config* directory does not exist, create it now). + 6. Select **Add** to add the new API signing key to your user settings. The Configuration File Preview dialog is displayed, containing a configuration file snippet with basic authentication information for the profile named ``DEFAULT`` (including the fingerprint of the API signing key you just created). + 7. Copy the configuration file snippet shown in the text box, and close the Configuration File Preview dialog. + 8. In a text editor, open the *~/.oci/config* file and paste the snippet into the file. (If the *~/.oci/config* does not exist, create it now). + 9. In the text editor, change the value of the ``key_file`` parameter of the profile to specify the path of the private key file (the *.pem* file you downloaded earlier). + 10. Save the changes you have made to the *~/.oci/config* file, and close the text editor. + 11. In a terminal window, change permissions on the private key file (the *.pem* file) to ensure that only you can read it, by entering: + ``chmod go-rwx ~/.oci/.pem`` + +Example of defining the model parameters: + .. code-block:: python + + llm = OCIGenAIModel( + model_id="", + service_endpoint="https://inference.generativeai..oci.oraclecloud.com", + compartment_id="ocid1.compartment.oc1.. Compartments in the OCI console website.>", + auth_type="API_KEY", + auth_profile="DEFAULT", + generation_config=generation_config, + ) + +Example of the key configuration in *.oci/config*: + .. code-block:: ini + + [DEFAULT] + user=ocid1.user.oc1.. + fingerprint= + tenancy=ocid1.tenancy.oc1.. + region= + key_file= + + This file is automatically generated and can be downloaded in step 7. + +OpenAI Model +------------ + +OpenAI Model is powered by `OpenAI `_. + +**Parameters** + +.. option:: model_id: str + + Name of the model to use. Current supported models: ``gpt-4o`` and ``gpt-4o-mini``. + +.. option:: generation_config: dict, optional + + Default parameters for text generation with this model. + +.. option:: proxy: str, optional + + Proxy settings to access the remote model. + +.. option:: api_type: OpenAIAPIType + + OpenAI API type to use. Should be one of the values from :ref:`OpenAIAPIType`. Defaults to `Chat Completions API `_ + +.. important:: + Ensure that the ``OPENAI_API_KEY`` is set beforehand + to access this model. A list of available OpenAI models can be found at + the following link: `OpenAI Models `_. + Among these, the supported models include ``gpt-4o`` and ``gpt-4o-mini``. + Note that the ``gpt-o1`` and ``gpt-o3`` models are not currently supported. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. openai-start + :end-before: .. openai-end + +.. collapse:: Equivalent code example utilizing the LlmModelFactory class. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. openai-llmfactory-start + :end-before: .. openai-llmfactory-end + +Gemini Model +------------ + +`Gemini `_ models can be accessed through +Google AI Studio or Google Vertex AI. + +**Parameters** + +.. option:: model_id: str + + Gemini model name as exposed by the selected service. For example, use the + model name listed in Google AI Studio when authenticating with + ``GeminiApiKeyAuth``, and use the model name listed in Vertex AI when + authenticating with ``GeminiCloudAuth``. + +.. option:: auth: GeminiApiKeyAuth | GeminiCloudAuth + + Required authentication configuration for Gemini. Use ``GeminiApiKeyAuth()`` if + you want the runtime to read ``GEMINI_API_KEY`` from the environment, or + ``GeminiCloudAuth(...)`` for Vertex AI. + +.. option:: generation_config: dict, optional + + Default parameters for text generation with this model. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. gemini-start + :end-before: .. gemini-end + +.. collapse:: Equivalent code example utilizing the LlmModelFactory class. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. gemini-llmfactory-start + :end-before: .. gemini-llmfactory-end + +**Notes** + +- Google AI Studio auth may read ``GEMINI_API_KEY`` from the environment. +- The important part is the service-side model name. AI Studio and Vertex AI may expose different Gemini model names or availability, so use the exact model name shown by the service you are connecting to. +- Vertex AI auth accepts an optional service-account JSON file path or an inline service-account JSON object. +- When ``vertex_credentials`` is omitted, LiteLLM/Google auth may rely on Application Default Credentials (ADC), such as ``GOOGLE_APPLICATION_CREDENTIALS``, the local ADC file created by ``gcloud auth application-default login``, or an attached service account. +- ADC does not always provide a default Vertex project, so ``project_id`` should usually still be set explicitly unless your environment already resolves it. + +vLLM Model +---------- + +`vLLM Model `_ is a model hosted with a vLLM server. + +**Parameters** + +.. option:: model_id: str + + Name of the model to use. + +.. option:: host_port: str + + Hostname and port of the vLLM server where the model is hosted. + +.. option:: generation_config: dict, optional + + Default parameters for text generation with this model. + +.. option:: api_type: OpenAIAPIType + + OpenAI API type to use. Should be one of the values from :ref:`OpenAIAPIType`. Defaults to `Chat Completions API `_ + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. vllm-start + :end-before: .. vllm-end + +.. collapse:: Equivalent code example utilizing the LlmModelFactory class. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. vllm-llmfactory-start + :end-before: .. vllm-llmfactory-end + +**Notes** + +Usually, vLLM models do not support tools calling. +To enable this functionality, WayFlow modifies the prompt by prepending and appending specific ReAct templates and formats tools accordingly when: + +- The model is required to utilize tools. +- The list of messages contains some ``tool_requests`` or ``tool_results``. + +Be aware of this when you generate with tools or tool calls. +To disable this behavior, set ``use_tools`` to ``False`` and ensure the prompt does not contain +``tool_call`` and ``tool_result`` messages. +See `this documentation `_ for more details on the ReAct prompting technique. + + +Ollama Model +------------ + +`Ollama Model `_ is powered by a locally hosted Ollama server. + +**Parameters** + +.. option:: model_id: str + + Name of the model to use. A list of model names can be found `here `_. + +.. option:: host_port: str + + Hostname and port of the Ollama server where the model is hosted. + By default Ollama binds port 11434. + +.. option:: generation_config: dict, optional + + Default parameters for text generation with this model. + +**Examples** + +.. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. ollama-start + :end-before: .. ollama-end + +.. collapse:: Equivalent code example utilizing the LlmModelFactory class. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. ollama-llmfactory-start + :end-before: .. ollama-llmfactory-end + +**Notes** + +As of November 2025, Ollama does not support `OpenAI Responses API `_. Therefore only Chat Completions API will be supported for OpenAI models. + +As of November 2024, Ollama does not support tools calling with token streaming. +To enable this functionality, WayFlow modifies the prompt by prepending and appending specific ReAct templates and formats tools accordingly when: + +- The model is required to utilize tools. +- The list of messages contains some ``tool_requests`` or ``tool_results``. + +Be aware of that when you generate with tools or tool calls. +To disable this behavior, set ``use_tools`` to ``False`` and ensure the prompt does not contain +``tool_call`` and ``tool_result`` messages. +See `this documentation `_ for more details on the ReAct prompting technique. + + +Troubleshooting +--------------- + +In certain situations, models may require messages to follow a specific order. +Message transforms such as :ref:`CanonicalizationMessageTransform ` can +be used to enforce the correct ordering of messages (for example, when working with Google models). +For additional details on LLM prompting, refer to the :doc:`Prompt Template guide `. + +Recap +----- + +This guide provides detailed descriptions of each model type supported by WayFlow, demonstrating how to use both the configuration dictionary and direct instantiation methods for each model. + +.. collapse:: Below is the complete code from this guide. + + .. literalinclude:: ../code_examples/example_initialize_llms.py + :language: python + :start-after: .. recap: + :end-before: .. end-recap + + +Next steps +---------- + +Having learned how to configure and initialize LLMs from different providers, you may now proceed to: + +- :doc:`Config Generation ` +- :doc:`How to Build Assistants with Tools ` + +Some additional resources we recommend are: + +- `HuggingFace - Generation with LLMs `_ +- `HuggingFace - Text generation strategies `_ diff --git a/26.1.2/_sources/core/index.rst.txt b/26.1.2/_sources/core/index.rst.txt new file mode 100644 index 000000000..5cad73832 --- /dev/null +++ b/26.1.2/_sources/core/index.rst.txt @@ -0,0 +1,124 @@ +WayFlow +======= + +*Robust AI-powered assistants for task automation and enhanced user experiences.* + +**WayFlow** is a powerful, intuitive Python library for building advanced AI-powered assistants. +It offers a standard library of modular building blocks to streamline the creation of both +workflow-based and agent-style assistants, encourages reusability and speeds up the development process. + +With WayFlow you can build both structured :ref:`Flows ` and autonomous :ref:`Agents `, giving you complete +flexibility and allowing you to choose the paradigm that best fits your use case. + +.. rubric:: Why WayFlow? + +WayFlow has several advantages over other existing open-source frameworks: + +* **Flexibility**: WayFlow supports multiple approaches to building AI Assistants, including Agents and Flows. +* **Interoperability**: WayFlow works with LLMs from many different vendors and supports an open approach to integration. +* **Reusability**: Build reusable and composable components to enable rapid development of AI Assistants. +* **Extensibility**: WayFlow has powerful abstractions to handle all types of LLM applications and provides a standard library of steps. +* **Openness**: We want to build a community and welcome contributions from diverse teams looking to take the next step in open-source AI Agents. + +.. rubric:: Quick start + +To install WayFlow from PyPI: + +.. only:: stable + + To install ``wayflowcore`` (on Python 3.10), use the following command to install it from PyPI: + + .. code-block:: bash + :substitutions: + + pip install "|package_name|==|stable_release|" + +.. only:: dev + + To install ``wayflowcore`` (on Python 3.10), use the following command to install it from source: + + .. code-block:: bash + :substitutions: + + bash install-dev.sh + +For full details on installation including what Python versions and platforms are supported please see our :doc:`installation guide`. + +With WayFlow installed, you can now try it out. + +WayFlow supports several LLM API providers. First choose an LLM from one of the options below: + +.. include:: _components/llm_config_tabs.rst + +Then create an agent and have a conversation with it, as shown in the code below: + +.. literalinclude:: code_examples/quickstart.py + :language: python + :start-after: .. full-code: + :end-before: .. end-full-code + +.. tip:: + **Self Hosted Models**: If you are interested in using locally hosted models, please see our guide on using them with WayFlow, :doc:`How to install Ollama `. + + +.. rubric:: Next Steps + +#. **Familiarize yourself with the basics - Tutorials** + + * Start with the :doc:`Tutorial building a simple conversational assistant with Agents `. + * Step through the :doc:`Tutorial building a simple conversational assistant with Flows `. + * And then do the :doc:`Tutorial building a simple code review assistant `. + +#. **Ways to use WayFlow to solve common tasks - How-to Guides** + + The :doc:`how-to guides ` show you how to achieve common tasks and use-cases using WayFlow. + They cover topics such as: + + * :doc:`How to create conditional transitions in Flows `. + * :doc:`How to build assistants with Tools `. + * :doc:`How to catch exceptions in flows `. + * :doc:`How to use Agents in Flows `. + * :doc:`How to connect assistants to data `. + +#. **Explore the API documentation** + + Dive deeper into the :doc:`API documentation ` to learn about the various classes, methods, and functions available in the + library. + +.. rubric:: Dive Deeper + +.. rubric:: Security + +LLM-based assistants and LLM-based flows require careful security assessments before deployment. +Please see our :doc:`Security considerations page ` to learn more. + +.. rubric:: Frequently Asked Questions + +Look through our :doc:`Frequently Asked Questions `. + +.. toctree:: + :hidden: + + Changelog + Installation + + +.. toctree:: + :hidden: + :caption: Essentials + + Tutorials & Use Cases + How-to Guides + Conceptual Guides + API Reference + +.. toctree:: + :hidden: + :caption: Resources + + Reference Sheet + Glossary + Security Considerations + For Contributors + Frequently Asked Questions + Docs MCP Server diff --git a/26.1.2/_sources/core/installation.rst.txt b/26.1.2/_sources/core/installation.rst.txt new file mode 100644 index 000000000..9ee3dbb5a --- /dev/null +++ b/26.1.2/_sources/core/installation.rst.txt @@ -0,0 +1,147 @@ +Installation +============ + +.. only:: stable + + You can find all versions and supported platforms of |project| in the :package_index:`\ `. + + For example, if you want to install |package_name| |stable_release|: + + .. code-block:: bash + :substitutions: + + pip install "|package_name|==|stable_release|" + + Installing with ``pip`` pulls prebuilt binary wheels on supported platforms. + + .. only:: builder_html + + The list below shows the package versions used in the CI environment, with Business Approval requests filed for each as part of the release process. + :download:`constraints.txt <../../../../wayflowcore/constraints/constraints.txt>` + + If you want to install |project| with exactly these package versions, download the file and run: + + .. code-block:: bash + :substitutions: + + pip install "|package_name|==|stable_release|" -c constraints.txt + +.. only:: dev + + 1. Clone the `repository `_. + + .. code-block:: bash + :substitutions: + + git clone git@github.com:oracle/wayflow.git + + .. tip:: + If you face any problem, check with the WayFlow team. + + Next, install WayFlow directly from source. + + 1. Create a fresh Python environment for building and running WayFlow assistants: + + .. code-block:: bash + :substitutions: + + python3.10 -m venv + source /bin/activate + + 2. Move to the *wayflowcore/wayflowcore* directory: + + .. code-block:: bash + :substitutions: + + cd wayflowcore/wayflowcore + + 3. Install ``wayflowcore``: + + .. code-block:: bash + :substitutions: + + bash install-dev.sh + + .. note:: + This removes any previous Python environment named ``venv-wayflowcore`` in order to create a new one. + +Supported Platforms +------------------- + +|project| strives for compatibility with major platforms and environments, where it is possible. + +Operation systems and CPU architectures +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :widths: 50 30 30 + :header-rows: 1 + + * - OS / CPU Architecture + - x86-64 Support + - ARM64 Support + * - Linux + - Supported + - Untested + * - MacOS + - Supported + - Supported + + +Python version +~~~~~~~~~~~~~~ + +.. list-table:: + :widths: 30 30 + :header-rows: 1 + + * - Python version + - Support + * - Python 3.9 + - Unsupported + * - Python 3.10 + - Supported + * - Python 3.11 + - Supported + * - Python 3.12 + - Supported + * - Python 3.13 + - Supported + * - Python 3.14 + - Supported + + +Package manager +~~~~~~~~~~~~~~~ + +.. list-table:: + :widths: 30 30 + :header-rows: 1 + + * - Package Manager + - Support + * - pip + - Supported + * - conda + - Untested + + +Python implementation +~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :widths: 30 30 + :header-rows: 1 + + * - Implementation + - Support + * - CPython + - Supported + * - PyPy + - Untested + +What do *Supported*, *Untested* and *Unsupported* mean? + +* *Unsupported*: The package or one of its dependencies is not compatible with the Python version. +* *Untested*: The package and its dependencies are compatible with the Python version, but they are not tested. +* *Supported*: The package and its dependencies are compatible with the Python version, and the package is tested on that version. diff --git a/26.1.2/_sources/core/misc/docs_mcp_server.rst.txt b/26.1.2/_sources/core/misc/docs_mcp_server.rst.txt new file mode 100644 index 000000000..c8b3cf2e2 --- /dev/null +++ b/26.1.2/_sources/core/misc/docs_mcp_server.rst.txt @@ -0,0 +1,306 @@ +.. _docs_mcp_server: + +=============== +Docs MCP Server +=============== + +WayFlow includes a documentation-focused MCP server in ``examples/mcp/docs_mcp.py``. +It exposes a single tool, ``get_docs``, that lets coding assistants safely navigate +the published WayFlow Markdown docs bundle and the generated end-to-end example files. + +This is useful when you want an assistant to answer questions grounded in the current +WayFlow documentation, inspect example code before writing new code, or locate the +right guide/API page without giving the assistant unrestricted shell access. + + +What the server does +==================== + +The ``get_docs`` tool accepts a constrained shell-like command and runs it against the +downloaded docs bundle. Supported commands are: + +* ``ls [PATH]`` or ``ls -la [PATH]`` +* ``rg --files [-g/--glob PATTERN] [PATH]`` +* ``sed -n 'START,ENDp' FILE`` +* ``head FILE`` or ``head -n N FILE`` +* ``tail FILE`` or ``tail -n N FILE`` +* exactly one pipe from a supported producer into ``head``, ``tail``, or filtering ``rg`` + +The server is intentionally narrow in scope: + +* it refreshes the Markdown bundle at startup from the base docs URL; +* it only allows navigation inside that bundle; +* it supports exactly one pipe and still blocks redirects, command chaining, and subshell syntax. + +The intended workflow is: + +1. discover folders with ``ls``; +2. locate the relevant page or example with ``rg --files`` or ``| rg`` filtering; +3. read or trim excerpts with ``sed -n``, ``head``, or ``tail``. + +Typical pipeline examples include: + +* ``rg --files core/howtoguides | rg mcp`` +* ``rg --files -g '*.md' core | head -n 20`` +* ``sed -n '1,220p' core/howtoguides/howto_mcp.md | rg StdioTransport`` + + +Adding the server to Codex +========================== + +The example file includes the minimal Codex setup instructions at the top. + +From the ``examples/mcp`` directory: + +.. code-block:: bash + + codex mcp add wayflow_docs -- python docs_mcp.py --base-docs-url https://oracle.github.io/wayflow/development + +If you prefer to edit the Codex configuration manually: + +.. code-block:: toml + + [mcp_servers.wayflow_docs] + command = "python" + args = ["docs_mcp.py", "--base-docs-url", "https://oracle.github.io/wayflow/development"] + + +If you launch Codex from the repository root instead, use +``examples/mcp/docs_mcp.py`` as the script path. + +If you prefer a ``uv``-based setup that does not require a local checkout of the +WayFlow repository, you can run the server directly from GitHub: + +.. code-block:: bash + + codex mcp add wayflow_docs -- uv run --with mcp -- https://raw.githubusercontent.com/oracle/wayflow/refs/heads/main/examples/mcp/docs_mcp.py --base-docs-url https://oracle.github.io/wayflow/development + +Equivalent Codex configuration: + +.. code-block:: toml + + [mcp_servers.wayflow_docs] + command = "uv" + args = [ + "run", "--with", "mcp", + "--", + "https://raw.githubusercontent.com/oracle/wayflow/refs/heads/main/examples/mcp/docs_mcp.py", + "--base-docs-url", + "https://oracle.github.io/wayflow/development", + ] + startup_timeout_sec = 10 + + #[mcp_servers.wayflow_docs.env] + #HTTPS_PROXY = "YOUR_PROXY_URL" + + +Using it from a WayFlow Agent +============================= + +The docs MCP server can also be consumed from WayFlow through :ref:`MCPTool ` +and the :ref:`StdioTransport `. + +The example below starts the docs MCP server as a local subprocess, connects a +single ``get_docs`` tool to an :ref:`Agent `, and lets the agent inspect the +docs before answering. + +.. code-block:: python + + from wayflowcore.agent import Agent + from wayflowcore.executors.executionstatus import UserMessageRequestStatus + from wayflowcore.mcp import MCPTool, StdioTransport, authless_mcp_enabled + from wayflowcore.models import VllmModel + + llm = VllmModel( + model_id="LLAMA_MODEL_ID", + host_port="LLAMA_API_URL", + ) + + docs_transport = StdioTransport( + command="python", + args=[ + "examples/mcp/docs_mcp.py", + "--base-docs-url", + "https://oracle.github.io/wayflow/development", + ], + cwd="/path/to/wayflow", + ) + + with authless_mcp_enabled(): + docs_tool = MCPTool( + name="get_docs", + client_transport=docs_transport, + ) + + assistant = Agent( + llm=llm, + tools=[docs_tool], + custom_instruction=( + "Use the get_docs tool to inspect the WayFlow docs and examples before " + "answering. Prefer ls, rg --files, head, tail, and short sed excerpts. " + "You may use a single pipe into head, tail, or filtering rg when helpful." + ), + ) + + conversation = assistant.start_conversation() + conversation.append_user_message( + "Find the MCP how-to and write a minimal stdio example for a WayFlow agent." + ) + status = conversation.execute() + + if isinstance(status, UserMessageRequestStatus): + print(conversation.get_last_message().content) + +.. note:: + + As with the rest of the :doc:`MCP guide <../howtoguides/howto_mcp>`, + ``authless_mcp_enabled()`` is for local and test usage only. + + +Example assistant queries +========================= + +Below are representative queries you can send to a code assistant equipped with the +``wayflow_docs.get_docs`` tool. The tool calls were exercised against the docs bundle +to illustrate the expected workflow and response style. + + +Example 1: Find the MCP guide and the recommended local transport +----------------------------------------------------------------- + +**User query** + +"Where is the MCP guide, and which transport should I use for a local subprocess server?" + +**Example tool calls** + +.. code-block:: text + + wayflow_docs.get_docs("rg --files core/howtoguides | rg mcp") + wayflow_docs.get_docs("head -n 40 core/howtoguides/howto_mcp.md") + wayflow_docs.get_docs("sed -n '740,820p' core/api/tools.md | rg StdioTransport") + +**Result the assistant would give** + +The MCP guide is in ``core/howtoguides/howto_mcp.md``. For a locally launched MCP +server, the docs recommend :ref:`StdioTransport `, which is intended +for subprocess-based communication on the same machine. The same guide also points to +the :doc:`Tools API <../api/tools>` for the transport details. + + +Example 2: Locate an end-to-end MCP example +------------------------------------------- + +**User query** + +"Show me an existing WayFlow example that connects an agent to an MCP server." + +**Example tool calls** + +.. code-block:: text + + wayflow_docs.get_docs("ls core/end_to_end_code_examples | rg howto_mcp") + wayflow_docs.get_docs("head -n 40 core/end_to_end_code_examples/howto_mcp.py") + +**Result the assistant would give** + +There is an end-to-end example in ``core/end_to_end_code_examples/howto_mcp.py``. +The relevant section shows a WayFlow ``Agent`` configured with MCP access, including +the transport setup, the MCP tool wiring, and a simple execution loop. + + +Example 3: Discover what documentation areas are available +---------------------------------------------------------- + +**User query** + +"What can you browse in the WayFlow docs bundle?" + +**Example tool call** + +.. code-block:: text + + wayflow_docs.get_docs("ls core | head -n 10") + +**Result the assistant would give** + +The bundle includes the main documentation areas such as ``api``, ``howtoguides``, +``tutorials``, ``misc``, and ``end_to_end_code_examples``, plus top-level pages like +``installation.md``, ``security.md``, and ``faqs.md``. A good next step is to narrow +the search to one area with ``rg --files`` and then read a short excerpt with ``sed``. + + +Example 4: Pull a glossary definition before writing code +--------------------------------------------------------- + +**User query** + +"Before you write code, remind me what WayFlow means by an Agent." + +**Example tool call** + +.. code-block:: text + + wayflow_docs.get_docs("head -n 40 core/misc/glossary.md") + +**Result the assistant would give** + +According to the glossary, an Agent is a conversational WayFlow assistant that can +interact with users, use tools, and collaborate with other WayFlow assistants to solve +requests. It is the right building block when you want an assistant to plan tool usage +flexibly instead of following a fixed flow. + + +Example 5: Ask the assistant to synthesize a stdio MCP snippet +-------------------------------------------------------------- + +**User query** + +"Use the docs to write a minimal Agent example that talks to an MCP server over stdio." + +**Example tool calls** + +.. code-block:: text + + wayflow_docs.get_docs("head -n 80 core/howtoguides/howto_mcp.md") + wayflow_docs.get_docs("sed -n '740,820p' core/api/tools.md | rg StdioTransport") + wayflow_docs.get_docs("head -n 40 core/end_to_end_code_examples/howto_mcp.py") + +**Result the assistant would give** + +Here is a minimal example based on the MCP how-to and the ``StdioTransport`` API: + +.. code-block:: python + + from wayflowcore.agent import Agent + from wayflowcore.executors.executionstatus import UserMessageRequestStatus + from wayflowcore.mcp import MCPTool, StdioTransport, authless_mcp_enabled + from wayflowcore.models import VllmModel + + llm = VllmModel( + model_id="LLAMA_MODEL_ID", + host_port="LLAMA_API_URL", + ) + + docs_transport = StdioTransport( + command="python", + args=[ + "examples/mcp/docs_mcp.py", + "--base-docs-url", + "https://oracle.github.io/wayflow/development", + ], + cwd="/path/to/wayflow", + ) + + with authless_mcp_enabled(): + docs_tool = MCPTool( + name="get_docs", + client_transport=docs_transport, + ) + + assistant = Agent( + llm=llm, + tools=[docs_tool], + ) + + ... diff --git a/26.1.2/_sources/core/misc/glossary.rst.txt b/26.1.2/_sources/core/misc/glossary.rst.txt new file mode 100644 index 000000000..2e2cb06bd --- /dev/null +++ b/26.1.2/_sources/core/misc/glossary.rst.txt @@ -0,0 +1,383 @@ +.. _core_ref_glossary: + +================ +WayFlow Glossary +================ + +This glossary introduces the key terms and concepts used across the WayFlow library. + +Assistant +========= + +WayFlow enables the creation of **AI-powered assistants** including: + +- :ref:`Flows ` which are structured assistants that follow a pre-defined task completion process; +- :ref:`Agents ` which are conversational assistants that can autonomously plan, think, act, and execute tools to complete tasks in a flexible manner. + +WayFlow assistants can be composed together and configured to solve complex tasks with varying degrees of autonomy, ranging from fully self-directed +to highly prescriptive, allowing for a spectrum of flexibility in task completion. + +See the :doc:`Tutorials and Use-Case Examples <../tutorials/index>` to learn to build WayFlow assistants. + +Agent +===== + +An :ref:`Agent ` is a type of LLM-powered assistant that can interact with users, leverage external tools, and interact with other WayFlow assistants +to take specific actions in order to solve user requests through conversational interfaces. + +A simple agentic system may involve a single Agent interacting with a human user. More advanced assistants may also involve the use of multiple agents. +Finally, Agents can be integrated in Flows with the :ref:`AgentExecutionStep `. + +To learn more about Agents, see the tutorial :doc:`Build a Simple Conversational Assistant with Agents <../tutorials/basic_agent>`, or read the :ref:`API reference `. + +Branching +========= + +Branching is the ability of a :ref:`Flow` to conditionally transition between different steps based on specific input values or conditions. +Developers can then create more dynamic and adaptive workflows that respond to varying scenarios. Branching is achieved through the use of the +:ref:`BranchingStep `, which defines multiple possible branches and maps input values to specific steps. + +Read the guide :doc:`How to Create Conditional Transitions in Flows <../howtoguides/conditional_flows>` for how to use branching. For more information, read the :ref:`API reference `. + +.. _defclienttool: + +Client Tool +=========== + +See :ref:`tools ` + +.. _context_section: + +Context Provider +================ + +:ref:`Context providers ` are callable components that are used to provide dynamic contextual information to WayFlow assistants. +They are useful to connect external datasources to an assistant. + +For instance, giving the current time of the day to an assistant can be achieved with a context provider. + +Read about the different types of context providers in the :ref:`API reference `. + +Control Flow Edge +================= + +A :ref:`Control Flow Edge ` is a connector that represents a directional link between two steps in a :ref:`Flow `. +It specifies a possible transition between a specific branch of a source step and a destination step. + +This concept enables assistant developers to explicitly define the expected transitions that can occur within a Flow. + +Read more about control flow edges in the :ref:`API reference `. + +Composability +============= + +Composability refers to the ability of WayFlow assistants to be decomposed into smaller components, combined with other components, and rearranged +to form new assistants that can solve a wide range of tasks. By supporting composability, you can create complex agentic systems +from simpler building blocks. + +WayFlow supports four types of agentic patterns: + +* Calling :doc:`Agents in Flows <../howtoguides/howto_agents_in_flows>`: Integrate conversational capabilities into structured workflows. +* Calling :ref:`Agents in Agents `: Combine multiple agents to execute complex tasks autonomously. +* Calling :ref:`Flows in Agents `: Use structured workflows as tools within conversational agents. +* Calling :ref:`Flows in Flows `: Create nested workflows to model complex business processes. + +Conversation +============ + +A :ref:`Conversation ` is a stateful object that represents the execution state of a WayFlow assistant. +It stores the list of :ref:`messages ` as well as information produced during the assistant execution (for example, tool calls, inputs/outputs produced by steps in a flow). + +The conversation object can be modified by the assistant through the ``execute`` method which updates the conversation state based on the assistant's logic. +It also serves as the interface from which end users can interact with WayFlow assistants (for example, by getting the current list of messages, appending a user message, and so on). + +The usual code flow when executing WayFlow assistants would be as follows: + +1. A new conversation is created using the ``start_conversation`` method from :ref:`Flows ` and :ref:`Agents `, with optional inputs. +2. Then in the main execution loop: + + * The user may interact with the assistant (for example, by adding a new message). + * The assistant execution is started/resumed with the ``execute`` method. + +For more information about how the Conversation is used, see the :doc:`Tutorials <../tutorials/index>`, or read the :ref:`API reference `. + + +Data Flow Edge +============== + +A :ref:`Data Flow Edge ` is a connector that represents a logical link between steps or context providers within a :ref:`Flow `. +It defines how data is propagated from the output of one step, or context provider, to the input of another step. + +This concept enables assistant developers to explicitly define the expected orchestration of data flows within Flows. + +Read more about data flow edges in the :ref:`API reference `. + +Execution Interrupts +==================== + +An :ref:`ExecutionInterrupt ` is a mechanism that allows assistant developers to intervene in the standard execution of an assistant, +providing the ability to stop or pause the execution when specific events or conditions are met, and execute a custom callback function in response. + +For example, the execution can be interrupted when a :ref:`time limit is reached ` or when a +:ref:`maximum number of tokens is exceeded `, triggering a callback to handle the interruption. + +Read more about execution interrupts in the :ref:`API reference `. + + +Execution Status +================ + +The :ref:`ExecutionStatus ` is a runtime indicator of an assistant's execution state in WayFlow. +This status provides information about the assistant's current activity, such as whether it has finished its execution, +is waiting for user input, or is waiting on a tool execution result from a :ref:`Client tool `. + +The ``ExecutionStatus`` is used in execution loops of :ref:`Agents ` and :ref:`Flows ` to properly manage the conversation with the assistant. + +Read more about the types of execution statuses and their use in the :ref:`API reference `. + +Flow +==== + +A :ref:`Flow ` is a type of structured assistant composed of individual :ref:`steps ` that are connected to form a coherent sequence of actions. +Each step in a Flow is designed to perform a specific function, similar to functions in programming. + +Flows can have loops, :ref:`conditional transitions `, and multiple end points. Flows can also :ref:`integrate sub-flows ` and +:ref:`Agents ` to enable more complex capabilities. + +A Flow can be used to tackle a wide range of business processes and other tasks in a controllable and efficient way. + +Read the tutorial how to :doc:`Build a Simple Fixed-Flow Assistant with Flows<../tutorials/basic_flow>`, or see the available :doc:`How-to Guides <../howtoguides/index>` about Flows. +Also, check the :ref:`API reference `. + +Generation Config +================= + +The :ref:`LLM generation config ` is the set of parameters that control the output of a :ref:`Large Language Model (LLM) ` in WayFlow. +These parameters include the maximum number of tokens to generate (``max_tokens``), the sampling ``temperature``, the probability threshold for nucleus sampling (``top_p``), and optional per-token log-probabilities (``top_logprobs``). + +Learn more about the LLM generation config in the :doc:`How to Specify the Generation Configuration when Using LLMs <../howtoguides/generation_config>` +or read the :ref:`API reference `. + +.. _llms_section: + +Large Language Model (LLM) +========================== + +A :ref:`Large Language Model ` is a type deep neural network trained on vast amounts of text data that can understand, generate, +and manipulate human language through pattern recognition and statistical relationships. It processes input text through multiple layers of neural networks, +using specific mechanisms to understand context and relationships between words. + +Modern LLMs contain billions of parameters and often require dedicated hardware for both training and inference. +As such, they are typically hosted through APIs by their respective providers, allowing for ease of integration and access. + +Notably, WayFlow does not handle the inference of LLMs on its server, instead relying on these external APIs to leverage the power of LLMs. +This approach allows WayFlow to remain lightweight while still providing access to the capabilities of these powerful models. + +Read our guide :doc:`How to Use LLMs from Different LLM Providers <../howtoguides/llm_from_different_providers>`, or see the :ref:`API reference ` +for the list of supported models. + +Message +======= + +A :ref:`Message ` is a core concept in WayFlow, representing a unit of communication between users and assistants. It provides a structured +way to hold information and can contain various types of data including text, :ref:`tool requests `, :ref:`results `, as well as other metadata. + +Messages are used throughout the library to hold information and facilitate communication between different components. +The list of messages generated during an assistant execution can be accessed directly from a :ref:`Conversation `. + +Read more about messages in the :ref:`API reference `. + +.. _prompt_engineering_section: + +Prompt Engineering and Optimization +=================================== + +Prompt engineering and optimization is the systematic process of designing, refining, and improving prompts to achieve more accurate, +reliable, and desired outputs from language models. It involves iterative testing and refinement of prompt structures, careful consideration of +context windows, and strategic use of examples and formatting. + +Methods such as Automated Prompt Engineering can help improve prompts by using algorithms to optimize the prompt performance on a specific metric. + +Prompt Engineering Styles +------------------------- + +.. list-table:: + :widths: 30 30 50 + :header-rows: 1 + + * - Technique + - Description + - Example + * - Zero-shot + - No example, just task + - "Summarize this article." + * - Few-shot + - Provide examples + - "Q: What is 2+2? A: 4..." + * - Chain-of-thought + - Encourage step-by-step thinking + - "Let's think step-by-step..." + * - Role prompting + - Assign a persona + - "You are an expert lawyer..." + * - Constraint-based + - Set strict formats or word limits + - Answer in JSON with keys 'title'..." + +Prompt Template +=============== + +A prompt template is a standardized prompt structure with placeholders for variable inputs, designed to maintain consistency across similar queries +while allowing for customization. WayFlow uses Jinja-style placeholders to specify the input variables to the prompt (for more information check +the `reference of Jinja2 `_). + +See the :doc:`Tutorials and Use-Case Examples <../tutorials/index>` for concrete examples, or check the +:ref:`TemplateRenderingStep API reference `. + +.. note:: + + Jinja templating introduces security concerns that are addressed by WayFlow by restricting Jinja's rendering capabilities. + Please check our guide on :ref:`How to write secure prompts with Jinja templating ` for more information. + +Prompt templates can be used in WayFlow components that use LLMs, such as :ref:`Agents ` and the :ref:`PromptExecutionStep `. + +Properties +========== + +A :ref:`Property ` is a metadata descriptor that provides information about an input/output value of a component (:doc:`Tools <../api/tools>`, +:ref:`Steps `, :ref:`Flows `, and :ref:`Agents `) in a WayFlow assistant. Properties can represent various data types such as +:ref:`boolean `, :ref:`float `, :ref:`integer `, :ref:`string `, +as well as nested types such as :ref:`list `, :ref:`dict `, or :ref:`object `. + +Properties include attributes such as name, description, and default value, which help to clarify the purpose and behavior of the component, +making it easier to understand and interact with the component. + +To learn more about the use of properties, read the guide :doc:`How to Change Input and Output Descriptors of Components<../howtoguides/io_descriptors>`, +and check the :ref:`API reference `. + + +Remote Tool +=========== + +See :ref:`tools ` + + +Retrieval Augmented Generation (RAG) +==================================== + +Retrieval Augmented Generation (RAG) is a technique to enhance LLM outputs by first retrieving relevant information from a knowledge base and then incorporating +it into the generation process. This approach enhances the model's ability to access and utilize specific information beyond its training data. + +RAG systems typically involves a retrieval component that searches for relevant information and a generation component that incorporates this +information into the final output. + +Serialization +============= +.. _defserialization: + +In WayFlow, serialization refers to the ability to capture the current configuration of an assistant and represent it in a compact, human-readable form. +This allows assistants to be easily shared, stored, or deployed across different environments, while maintaining their functionality and consistency. + +Read the guide :doc:`How to Serialize and Deserialize Flows and Agents <../howtoguides/howto_serdeser>` or check the :ref:`API reference ` for more information. + +.. _defservertool: + +Server Tool +=========== + +See :ref:`tools ` + +Step +==== + +A :ref:`Step ` is an atomic element of a :ref:`Flow ` that encapsulates a specific piece of logic or functionality. +WayFlow proposes a variety of steps with functionalities ranging from :ref:`LLM generation ` and :ref:`tool use ` +to :ref:`branching `, :ref:`data extraction `, and :ref:`much more `. By composing the steps together, WayFlow enables +the creation of powerful structured assistants to solve diverse use cases efficiently and reliably. + +Check the list of available steps in the :ref:`API reference `. + +.. _defstructuredgeneration: + +Structured Generation +===================== + +Structured generation is the process of controlling LLM outputs to conform to specific formats, schemas, or patterns, ensuring consistency and +machine-readability of generated content. It involves techniques for guiding the model to produce outputs that follow predetermined structures +while maintaining natural language fluency. + +This approach is particularly valuable for generating data in formats like JSON, XML, or other structured representations. + +For more information, see the guide :doc:`How to Do Structured LLM Generation in Flows <../howtoguides/howto_promptexecutionstep>`. + +.. _tools_section: + +Tools +===== + +.. image:: ../_static/howto/types_of_tools.svg + :align: center + :scale: 100% + :alt: Types of tools + + +WayFlow support three types of tools: + +Server Tool +----------- + +A :ref:`Server Tool ` is the simplest type of tool available in WayFlow. It is simply defined with the signature of the tool to execute including: + +* A tool name. +* A tool description. +* The names, types, and optional default values for the input parameters of the tool. +* A Python callable, which is the callable to invoke upon the tool execution. +* The output type. + +See the guide :doc:`How to Build Assistants with Tools <../howtoguides/howto_build_assistants_with_tools>` for how to use Server tools. +For more information about the ``ServerTool``, read the :ref:`API reference `. + + +Client Tool +----------- + +A :ref:`Client tool ` is a type of tool that can be built in WayFlow. Contrary to the :ref:`Server Tool ` which is directly executed on the server side, +upon execution the client tool returns a :ref:`ToolRequest ` to be executed on the client side, which then sends the execution result back to the assistant. + +See the guide :doc:`How to Build Assistants with Tools <../howtoguides/howto_build_assistants_with_tools>`. For more information about the ``ClientTool``, read the :ref:`API reference `. + + +Remote Tool +----------- + +A :ref:`Remote tool ` is a type of tool that can be used in WayFlow to perform API calls. + +For more information about the ``RemoteTool``, read the :ref:`API reference `. + +Tokens +====== + +Tokens are the fundamental units of text processing in LLMs, representing words, parts of words, or characters that the model uses to understand and generate language. + +They form the basis for the model's context window size and directly impact processing costs and performance. + +It is worth noting that there are two types of tokens relevant to LLMs: input tokens and output tokens. +Input tokens refer to the tokens that are fed into the model as input, whereas output tokens are the tokens generated by the model as output. +In general, output tokens are more expensive than input tokens. An example of pricing can be $3 per 1M input tokens, and $10 per 1M output tokens. + +.. _variables_section: + +Variable +======== + +A :ref:`Variable ` is a flow-scoped data container that enables the storage and retrieval of data throughout a :ref:`Flow `. +Variables act as the shared state or context of a Flow (often referred to as `state` in other frameworks), +providing a way to decouple data from specific steps and make it available for use within multiple parts of the Flow. +They can be accessed and modified throughout a Flow using :ref:`VariableStep `, :ref:`VariableReadStep `, +and :ref:`VariableWriteStep ` operations. +:ref:`VariableStep ` provides an API to read, write, or update (write and then read) multiple variables in a single step. +:ref:`VariableReadStep ` and :ref:`VariableWriteStep ` are simpler steps used exclusively to read from or write to a single variable, respectively. + +Note that Variables are complementary to the value stored in the input/output dictionary which is specific to the steps execution. + +Learn more about Variables in the :ref:`API reference `. diff --git a/26.1.2/_sources/core/misc/reference_sheet.rst.txt b/26.1.2/_sources/core/misc/reference_sheet.rst.txt new file mode 100644 index 000000000..3b5e7afd1 --- /dev/null +++ b/26.1.2/_sources/core/misc/reference_sheet.rst.txt @@ -0,0 +1,558 @@ +.. _core_ref_sheet: + +======================= +WayFlow Reference Sheet +======================= + +This reference sheet provides a single-page overview of basic code snippets covering the core concepts used in WayFlow. + +Each section includes links to additional tutorials and guides for deeper learning. + +LLMs +==== + +WayFlow Agents and Flows may require the use of Large Language Models (LLMs). +This section shows how to initialize an LLM and perform quick tests. + + +Loading an LLM instance +----------------------- + +WayFlow supports several LLM API providers. +For an overview of supported LLMs, see the guide +:doc:`How to Use LLMs from Different Providers <../howtoguides/llm_from_different_providers>`. + +Start by selecting an LLM from one of the available providers: + +.. include:: ../_components/llm_config_tabs.rst + +Read more about the LLMs support in the :ref:`API reference `. + + +Testing inference with LLMs +--------------------------- + +Single prompt generation +^^^^^^^^^^^^^^^^^^^^^^^^ + +Use a simple :ref:`PromptExecutionStep ` to test an LLM. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-single_generation: + :end-before: .. end-single_generation + +**API Reference:** :ref:`PromptExecutionStep ` | :ref:`Flow ` + +.. tip:: + + Use the helper methods ``create_single_step_flow`` and ``run_flow_and_return_outputs`` for quick prototyping. + + +Parallel generation +^^^^^^^^^^^^^^^^^^^ + +Add a :ref:`MapStep ` to perform inference on a batch of inputs (parallel generation). + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-parallel_generation: + :end-before: .. end-parallel_generation + +**API Reference:** :ref:`ListProperty ` | :ref:`PromptExecutionStep ` | :ref:`MapStep ` | :ref:`Flow ` + +.. note:: + + Note the use of a :ref:`ListProperty ` to specify the output of the :ref:`MapStep `. + + +Structured generation +^^^^^^^^^^^^^^^^^^^^^ + +WayFlow supports :ref:`structured generation ` (such as controlling LLM outputs to conform to specific formats, schemas, or patterns, for example, Json Schema). + +Structured generation can be achieved by specifying the output descriptors of the :ref:`PromptExecutionStep `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-structured_generation: + :end-before: .. end-structured_generation + +**API Reference:** :ref:`StringProperty ` | :ref:`PromptExecutionStep ` | :ref:`Flow ` + +Read the guide on :doc:`How to Perform Structured Generation <../howtoguides/howto_promptexecutionstep>` for more information. + + +Tools +===== + +Tools are essential for building powerful Agents and Flows. +WayFlow supports the use of :ref:`ServerTool ` (which can be simply built with the :ref:`tool ` decorator), the :ref:`RemoteTool ` as well as +the :ref:`ClientTool `. + +.. image:: ../_static/howto/types_of_tools.svg + :align: center + :scale: 60% + +**Figure:** The different tools in WayFlow. + + +Creating a simple tool +---------------------- + +The simplest way to create a tool in WayFlow is by using the :ref:`tool ` decorator, which creates a :ref:`ServerTool ` (see definition :ref:`in the glossary `). + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-simple_server_tool: + :end-before: .. end-simple_server_tool + +**API Reference:** :ref:`tool ` + +For more information, read the guide on :doc:`How to Build Assistants with Tools <../howtoguides/howto_build_assistants_with_tools>` or read +the :doc:`API reference <../api/tools>` to learn about the available types of tools in WayFlow. + + +Creating a stateful tool +------------------------ + +To build stateful tools, simply use the :ref:`tool ` helper as a wrapper to the method of an instantiated class. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-simple_stateful_tool: + :end-before: .. end-simple_stateful_tool + +**API Reference:** :ref:`tool ` + + +Creating and using a Client tool +-------------------------------- + +Use the :ref:`ClientTool ` to create tools that are meant to be executed on the client side (see :ref:`definition in the glossary `). + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-client_tool: + :end-before: .. end-client_tool + +**API Reference:** :ref:`ClientTool ` | :ref:`ToolRequest ` | :ref:`ToolResult ` | :ref:`ToolExecutionStep ` | :ref:`ToolRequestStatus ` | :ref:`Flow ` + +Learn more about tools by reading :doc:`How to Build Assistants with Tools <../howtoguides/howto_build_assistants_with_tools>`, and the :doc:`Tools API reference <../api/tools>`. + +.. _refsheet_executionloop: + +Execution loop and statuses +=========================== + +This section illustrates a basic execution loop for WayFlow assistants (Agents and Flows). + +1. A new conversation is created. +2. The assistant is executed on the conversation. +3. Based on the status returned from the assistant execution: + * The loop exits if the status is ``FinishedStatus``. + * The user is prompted for input if the status is ``UserMessageRequestStatus``. + * A ``ClientTool`` is executed if the status is ``ToolRequestStatus``. + +The loop continues until the assistant returns a ``FinishedStatus``. + +.. code:: python + + from typing import Any + from wayflowcore.messagelist import Message, MessageType + from wayflowcore.executors.executionstatus import ( + FinishedStatus, UserMessageRequestStatus, ToolRequestStatus + ) + from wayflowcore.tools import ToolRequest, ToolResult + + def execute_client_tool_from_tool_request(tool_request: ToolRequest) -> Any: + if tool_request.name == "my_tool_name": + return _my_tool_callable(**tool_request.args) + else: + raise ValueError(f"Tool name {tool_request.name} is not recognized") + + conversation_inputs = {} + conversation = assistant.start_conversation(inputs=conversation_inputs) + + while True: + status = conversation.execute() + assistant_reply = conversation.get_last_message() + if assistant_reply: + print(f"Assistant>>> {assistant_reply.content}\n") + + if isinstance(status, FinishedStatus): + print(f"Finished assistant execution. Output values:\n{status.output_values}",) + break + elif isinstance(status, UserMessageRequestStatus): + user_input = input("User>>> ") + print("\n") + conversation.append_user_message(user_input) + elif isinstance(status, ToolRequestStatus): + tool_request = status.tool_requests[0] + tool_result = execute_client_tool_from_tool_request(tool_request) + print(f"{tool_result!r}") + conversation.append_message( + Message( + tool_result=ToolResult(content=tool_result, tool_request_id=tool_request.tool_request_id), + message_type=MessageType.TOOL_RESULT, + ) + ) + else: + raise ValueError(f"Unsupported execution status: '{status}'") + + +Learn more about execution loops by reading the :ref:`Execution Status API reference `. + +Agents +====== + +WayFlow :ref:`Agents ` are LLM-powered assistants that can interact with users, leverage external tools, and interact with other WayFlow assistants to take specific actions +in order to solve user requests through conversational interfaces. + +Creating a simple Agent +----------------------- + +Creating an :ref:`Agent ` only requires an LLM and optional instructions to guide the agent behavior. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-simple_agent: + :end-before: .. end-simple_agent + +**API Reference:** :ref:`Agent ` + +Learn more about Agents by reading the tutorial :doc:`Build a Simple Conversational Assistant with Agents <../tutorials/basic_agent>` +and the :ref:`Agent API reference `. + + +Creating a Agent with tools +--------------------------- + +You can simply equip :ref:`Agents ` with tools using the ``tools`` attribute of the agent. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-agent_with_tool: + :end-before: .. end-agent_with_tool + +**API Reference:** :ref:`Agent ` | :ref:`tool ` + + +Flows +===== + +WayFlow :ref:`Flows ` are LLM-powered structured assistants composed of individual steps that are connected to form a coherent sequence of actions. +Each step in a ``Flow`` is designed to perform a specific function, similar to functions in programming. + + +Creating a simple Flow +---------------------- + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-simple_flow: + :end-before: .. end-simple_flow + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`Flow ` | :ref:`OutputMessageStep ` + +Learn more about Flows by reading the tutorial :doc:`Build a Simple Fixed-Flow Assistant with Flows <../tutorials/basic_flow>` and the :ref:`Flow API reference `. + + +Creating Flow with explicit data connection +------------------------------------------- + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_with_dataconnection: + :end-before: .. end-flow_with_dataconnection + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`OutputMessageStep ` + +Learn more about data flow edges in the :ref:`Data Flow Edges API reference `. + + +Executing a sub-flow to an iterable with the MapStep +---------------------------------------------------- + +Applying or executing a sub-flow to an iterable is a common pattern and can be achieved in WayFlow using the :ref:`MapStep `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_with_mapstep: + :end-before: .. end-flow_with_mapstep + +**API Reference:** :ref:`Flow ` | :ref:`MapStep ` | :ref:`OutputMessageStep ` | :ref:`AnyProperty ` + +Learn more about MapSteps by reading :doc:`How to Do Map and Reduce Operations in Flows <../howtoguides/howto_mapstep>` and the :ref:`MapStep API reference `. + + +Adding conditional branching to Flows with the BranchingStep +------------------------------------------------------------ + +It is also frequent to want to transition in a :ref:`Flow ` depending on a condition, and this can be achieved in WayFlow with the :ref:`BranchingStep `. + +.. image:: ../_static/howto/branchingstep.svg + :align: center + :scale: 95% + :alt: Flow diagram of a simple branching step + +**Figure:** An example of a ``Flow`` using a ``BranchingStep``. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_with_branching: + :end-before: .. end-flow_with_branching + +**API Reference:** :ref:`Flow ` | :ref:`BranchingStep ` | :ref:`OutputMessageStep ` + +Learn more about branching steps by reading :doc:`How to Create Conditional Transitions in Flows <../howtoguides/conditional_flows>` and +the :ref:`BranchingStep API reference `. + + +Adding tools to Flows with the ToolExecutionStep +------------------------------------------------ + +To use tools in :ref:`Flows `, use the :ref:`ToolExecutionStep `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_with_tools: + :end-before: .. end-flow_with_tools + +**API Reference:** :ref:`Flow ` | :ref:`ToolExecutionStep ` | :ref:`ServerTool ` + +Learn more about ``ToolexecutionSteps`` by reading :doc:`How to Build Assistants with Tools <../howtoguides/howto_build_assistants_with_tools>` +and the :ref:`ToolexecutionSteps API reference `. + + +Agentic composition patterns +============================ + +There are four majors agentic composition patterns supported in WayFlow: + +* Calling Agents in Flows +* Calling Agents in Agents +* Calling Flows in Agents +* Calling Flows in Flows + + +Using an Agent in a Flow +------------------------ + +To use :ref:`Agents ` in :ref:`Flows `, you can use the :ref:`AgentExecutionStep `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-agent_in_flow: + :end-before: .. end-agent_in_flow + +**API Reference:** :ref:`Flow ` | :ref:`Agent ` | :ref:`AgentExecutionStep ` + +Learn more about Agents in Flows by reading :doc:`How to Use Agents in Flows <../howtoguides/howto_agents_in_flows>` +and the :ref:`Agent Execution Step API reference `. + +.. warning:: + + The ``AgentExecutionStep`` is currently in beta and may undergo significant changes. + The API and behaviour are not guaranteed to be stable and may change in future versions. + + +Multi-Level Agent Workflows +--------------------------- + +WayFlow supports hierarchical multi-agent systems, by using expert :ref:`Agents ` with a master / manager agent. +This can be achieved by using a :ref:`DescribedAgent `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-agent_in_agent: + :end-before: .. end-agent_in_agent + + +**API Reference:** :ref:`Agent ` | :ref:`DescribedAgent ` + +.. warning:: + + The use of expert agents is currently in beta and may undergo significant changes. + The API and behaviour are not guaranteed to be stable and may change in future versions. + + +Using Flows Within Agents +------------------------- + +To use :ref:`Flows ` in :ref:`Agents `, use the :ref:`DescribedFlow `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_in_agent: + :end-before: .. end-flow_in_agent + + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`PromptExecutionStep ` | :ref:`DescribedFlow ` + +Learn more about the use of :ref:`Flows ` in :ref:`Agents ` in the :ref:`API reference `. + + +Using Sub-Flows Within Flows +---------------------------- + +To use sub-flows in :ref:`Flows `, use the :ref:`FlowExecutionStep `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_in_flow: + :end-before: .. end-flow_in_flow + + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`FlowExecutionStep ` | :ref:`OutputMessageStep ` | :ref:`PromptExecutionStep ` + +Learn more about the use of sub-flows in :ref:`Flows ` by reading the :ref:`FlowExecutionStep API reference `. + + +Saving and loading WayFlow assistants +===================================== + +.. image:: ../_static/howto/ser_deser.svg + :align: center + :scale: 100% + :alt: Serialization/deserialization of Agents and Flows in WayFlow + +**Figure:** How serialization works in WayFlow. + + +Saving and loading simple assistants +------------------------------------ + +Save and load WayFlow assistants using the :ref:`serialize ` and :ref:`autodeserialize ` helper functions. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-serialize_simple_assistants: + :end-before: .. end-serialize_simple_assistants + +**API Reference:** :ref:`Agent ` | :ref:`serialize ` | :ref:`autodeserialize ` + +Learn more about Serialisation by reading :doc:`How to Serialize and Deserialize Flows and Agents <../howtoguides/howto_serdeser>` +and the :ref:`Serialisation API reference `. + + +Saving and loading assistants with tools +---------------------------------------- + +Register tools to a ``DeserializationContext`` to load assistants using tools. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-serialize_assistants_with_tools: + :end-before: .. end-serialize_assistants_with_tools + +**API Reference:** :ref:`Agent ` | :ref:`serialize ` | :ref:`autodeserialize ` | :ref:`tool ` + +Learn more about Serialisation by reading :doc:`How to Serialize and Deserialize Flows and Agents <../howtoguides/howto_serdeser>` +and the :ref:`Serialisation API reference `. + + +Providing context to assistants +=============================== + +Passing contextual information to assistants can be done in several ways, including: + +* By specifying input values when creating the :ref:`Conversation `. +* By using :ref:`ContextProviders `. +* By using :ref:`Variables `. + + +Providing context with inputs +----------------------------- + +You can pass static inputs when creating a new :ref:`Conversation `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-inputs_provider: + :end-before: .. end-inputs_provider + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`Flow ` | :ref:`OutputMessageStep ` + +Learn more about passing static inputs in the :ref:`Conversation API reference `. + +Providing dynamic inputs with ContextProviders +---------------------------------------------- + +:ref:`ContextProviders ` can be used to provide dynamic information to WayFlow assistants. + +Using the ToolContextProvider +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use the :ref:`ToolContextProvider ` to provide information to an assistant with a tool. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-tool_contextprovider: + :end-before: .. end-tool_contextprovider + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`ToolContextProvider ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`tool ` | :ref:`OutputMessageStep ` + +Learn more by reading the :ref:`ToolContextProvider API reference `. + + +Using the FlowContextProvider +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use the :ref:`FlowContextProvider ` to provide information to an assistant with a :ref:`Flow `. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-flow_contextprovider: + :end-before: .. end-flow_contextprovider + +**API Reference:** :ref:`FlowContextProvider ` | :ref:`ControlFlowEdge ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`OutputMessageStep ` + +Learn more by reading the :ref:`FlowContextProvider API reference `. + + +Using Variables to provide context +---------------------------------- + +You can use :ref:`Variables ` as an alternative way to manage context (or shared state) in :ref:`Flows `. +They let you store and reuse information across different steps in Flows. + +.. literalinclude:: ../code_examples/reference_sheet.py + :language: python + :start-after: .. start-context_with_variables: + :end-before: .. end-context_with_variables + +**API Reference:** :ref:`ControlFlowEdge ` | :ref:`DataFlowEdge ` | :ref:`Flow ` | :ref:`ListProperty ` | :ref:`FloatProperty ` | :ref:`VariableStep ` | :ref:`VariableReadStep ` | :ref:`VariableWriteStep ` | :ref:`OutputMessageStep ` | :ref:`Variable ` + +Learn more by reading the :ref:`Variables API reference `. + + + +.. _flowbuilder_ref_sheet: + +Flow Builder quick snippets +--------------------------- + +Build a sequence, then entry/finish: + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_linear_flow + :end-before: .. end-##_Build_a_linear_flow + +API Reference: :ref:`FlowBuilder ` + +Build a linear flow in one line: + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_linear_flow_equivalent + :end-before: .. end-##_Build_a_linear_flow_equivalent + + +Add a conditional using a step output as key, with a default branch: + +.. literalinclude:: ../code_examples/howto_flowbuilder.py + :language: python + :start-after: .. start-##_Build_a_flow_with_a_conditional + :end-before: .. end-##_Build_a_flow_with_a_conditional diff --git a/26.1.2/_sources/core/misc/type_casting_rules.rst.txt b/26.1.2/_sources/core/misc/type_casting_rules.rst.txt new file mode 100644 index 000000000..93d513bb1 --- /dev/null +++ b/26.1.2/_sources/core/misc/type_casting_rules.rst.txt @@ -0,0 +1,12 @@ +:orphan: + +.. csv-table:: + :header: "From \ To", "String", "Number", "Integer", "Object", "Array", "Boolean", "Null" + + "**String**", "✅", "❌", "❌", "❌", "❌", "❌", "❌" + "**Number**", "✅ json.dumps(value)", "✅", "✅ decimals truncated", "❌", "❌", "✅ value != 0", "❌" + "**Integer**", "✅ json.dumps(value)", "✅", "✅", "❌", "❌", "✅ value != 0", "❌" + "**Object**", "✅ json.dumps(value)", "❌", "❌", "✅", "❌", "❌", "❌" + "**Array**", "✅ json.dumps(value)", "❌", "❌", "❌", "✅", "❌", "❌" + "**Boolean**", "✅ json.dumps(value)", "✅ 1 if value else 0", "✅ 1 if value else 0", "❌", "❌", "✅", "❌" + "**Null**", "✅ empty string", "❌", "❌", "❌", "❌", "❌", "✅" diff --git a/26.1.2/_sources/core/rfcs-template.rst.txt b/26.1.2/_sources/core/rfcs-template.rst.txt new file mode 100644 index 000000000..fce601df7 --- /dev/null +++ b/26.1.2/_sources/core/rfcs-template.rst.txt @@ -0,0 +1,49 @@ +:orphan: + +======================== +WayFlow RFC Template +======================== + +Use this template to propose a **new feature**, **design change**, or **enhancement** to WayFlow. +Please keep it concise but thorough so reviewers can understand the motivation, scope, and impact of your idea. + +Summary +------- + +One-liner summary of the proposal. +What problem does this solve or what capability does it add? + +Motivation +---------- + +Explain the problem or opportunity. + +- What is the use case? +- What is the impact on end users or developers? + +Proposal +-------- + +Describe the solution in detail. + +- How do you propose to solve it? +- Include high-level architecture or code snippets if helpful. +- What components of the project are affected? + +Potential risks or concerns +--------------------------- + +Are there any drawbacks or compatibility concerns to be aware of? + +- Security, performance, or maintainability concerns? +- Backward compatibility? + +Related Links (Optional) +------------------------ + +Include links to relevant GitHub issues, external references, docs, etc. + +Implementation Plan (Optional) +------------------------------ + +If you already have an idea of the implementation steps, outline them here. diff --git a/26.1.2/_sources/core/security.rst.txt b/26.1.2/_sources/core/security.rst.txt new file mode 100644 index 000000000..42ff162bc --- /dev/null +++ b/26.1.2/_sources/core/security.rst.txt @@ -0,0 +1,658 @@ +Security Considerations +======================= + +Scope: WayFlow Agents, Flows, Steps, or Tools in any environment (development, staging, production). + +Why it matters: WayFlow integrates LLMs, Python, tools, and APIs. Its attack surface spans all Steps, Agents, Flows, Tools, and Context Providers. A single untrusted component (e.g., an unsafe tool or poorly-scoped ``ContextProvider``) can compromise the entire runtime. + +Considerations regarding tools +------------------------------- + +WayFlow allows LLMs to interact with applications via **Tools** (:ref:`ClientTool `, :ref:`ServerTool `, or :ref:`RemoteTool `), granting access to state or operations. Since tool inputs often come from LLMs or users, rigorous input sanitization and validation are crucial. + +**Key Principles for Tool Security:** + +* **Mandatory Input Validation**: Always validate tool inputs. + + * For :ref:`ClientTool ` and :ref:`ServerTool `, use ``input_descriptors`` to define/enforce schemas (types and descriptions) as a primary defense. Note that ``input_descriptors`` do not support constraints like ranges or max lengths, so you must implement additional validation in your tool's code. + * For :ref:`RemoteTool `, which constructs HTTP requests, validation is critical for all parameters that can be templated (e.g., ``url``, ``method``, ``data``, ``params``, ``headers``, ``cookies``). While ``input_descriptors`` can define the schema for arguments *passed to* the :ref:`RemoteTool `, the core security lies in controlling how these arguments are used to form the request and in features like ``url_allow_list``. +* **Output Scrutiny**: Define expected outputs with ``output_descriptors``. For :ref:`ClientTool `, clients post results; for :ref:`ServerTool `, the server ``func`` generates them. Calling Flows/Agents must validate and sanitize incoming :ref:`ToolResult ` content before relying on it. + + * When prompt templates or model adapters pass :ref:`ToolResult ` data back to an LLM, prefer a structured encoding instead of raw free text whenever possible. + * Keep tool-returned data clearly separated from system, developer, or user instructions. +* **Least Privilege**: Grant tools only permissions essential for their function. + +Tool Security Specifics +~~~~~~~~~~~~~~~~~~~~~~~ + +.. important:: + + Rigorously sanitize all tool inputs. Tools, bridging LLM understanding with system operations, are prime targets. Validate types, lengths, and semantic correctness before core logic execution. + +**`ServerTool` Considerations:** + +* **Callable Security (`func`)**: The ``func`` callable in :ref:`ServerTool ` is the primary security concern. Harden this server-executed code against vulnerabilities. +* **Isolation for High-Risk Tools**: Run high-risk :ref:`ServerTool ` instances (e.g., with elevated permissions, network/filesystem access) in sandboxed environments (containers/pods) with minimal IAM roles. Deny network/filesystem writes unless essential. +* **Tools from `Flows` or `Steps`**: + + * When creating a :ref:`ServerTool ` via :meth:`~wayflowcore.tools.servertools.ServerTool.from_flow`, or :meth:`~wayflowcore.tools.servertools.ServerTool.from_step`, its security is inherited. Ensure source ``Flow`` or ``Step`` tools are secure. + +**`ClientTool` Considerations:** + +* **Client-Side Execution**: :ref:`ClientTool ` execution occurs on the client. Client environment security, though outside WayFlow's control, impacts overall application security. +* **Untrusted `ToolRequest`**: Clients receive a :ref:`ToolRequest `. Though WayFlow generates ``name`` and ``tool_request_id``, client code must parse ``args`` with a strict schema, avoiding direct use in shell commands or sensitive OS functions. +* **Untrusted Client `ToolResult`**: Server-side WayFlow components must treat :ref:`ToolResult ` from clients as untrusted. Validate its ``content`` before processing. + +**`RemoteTool` Considerations:** + +* **Templated Request Arguments**: :ref:`RemoteTool ` allows various parts of the HTTP request (URL, method, body, headers, etc.) to be templated using Jinja. This is powerful but introduces risks if the inputs to these templates are not strictly controlled. Maliciously crafted inputs could lead to information leakage (e.g., exposing sensitive data in URLs or headers) or enable attacks like SSRF (Server-Side Request Forgery) or automated DDoS. Prefer a fixed developer-controlled base URL and template only path, query, or body values. +* **URL Allow List (`url_allow_list`)**: This is a critical security feature. We strongly recommend defining a ``url_allow_list`` to restrict the tool to a predefined set of allowed URLs or URL patterns. This significantly mitigates the risk of the tool being used to make requests to unintended or malicious endpoints. In code, ``url_allow_list`` is required only when placeholders appear in the destination part of the URL (scheme, host, or port). Placeholders limited to the path or query do not trigger this requirement. Refer to the API documentation for detailed matching rules. +* **Secure Connections (`allow_insecure_http`)**: By default, :ref:`RemoteTool ` disallows non-HTTPS URLs (``allow_insecure_http=False``). Maintain this default unless there's an explicit, well-justified reason to allow insecure HTTP, and ensure the risks are understood. +* **Credential Handling (`allow_credentials`)**: By default (``allow_credentials=True``), URLs can contain credentials (e.g., ``https://user:pass@example.com``). If your use case does not require this, set ``allow_credentials=False`` to prevent accidental leakage or misuse of credentials in URLs. +* **URL Fragments (`allow_fragments`)**: Control whether URL fragments (e.g., ``#section``) are permitted in requested URLs and allow list entries. Default is ``True``. Set to ``False`` if fragments are not needed and could introduce ambiguity or bypass attempts. +* **Non-Public IP Targets**: Requests to loopback, link-local, or private IP literal targets that are not explicitly allow-listed emit a runtime warning. Treat this as a sign to double-check that the destination is expected and properly restricted. +* **Output Parsing (`output_jq_query`)**: If using ``output_jq_query`` to parse JSON responses, be aware that complex queries on very large or maliciously structured JSON could consume significant resources. While primarily a performance concern, extreme cases might have denial-of-service implications. + +.. caution:: + + As highlighted in the :ref:`RemoteTool ` API, since the Agent can generate arguments + (url, method, data, params, headers, cookies) or parts of these arguments in the respective + Jinja templates, this can impose a security risk of information leakage and enable specific attack + vectors like automated DDOS attacks. Please use :ref:`RemoteTool ` responsibly and ensure + that only valid URLs can be given as arguments or that no sensitive information is used for any of these + arguments by the agent. Utilize ``url_allow_list``, ``allow_credentials``, and ``allow_fragments`` + to control URL validity. ``url_allow_list`` is required only when placeholders appear in the + destination part of the URL (scheme, host, or port), not when placeholders are limited to the + path or query. + +Harden All Tools (ServerTool, ClientTool and RemoteTool) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: Tool Hardening Guidelines + :widths: 30 70 + :header-rows: 1 + + * - Risk + - Guidance + * - Unvalidated arguments (leading to injection, DoS, etc.) + - **Primary Defense**: + + * For :ref:`ClientTool ` & :ref:`ServerTool `: Use ``input_descriptors`` for basic type checking. In :ref:`ServerTool `'s ``func``, add comprehensive validation (string length limits, numeric ranges, format constraints). Cap string lengths and numeric ranges in tool implementation code. + * For :ref:`RemoteTool `: + + * Rigorously validate and sanitize any inputs used in templated arguments (``url``, ``method``, ``data``, ``params``, ``headers``, ``cookies``). + * Keep the base URL developer-controlled and template only path, query, or body values whenever possible. + * Strongly prefer configuring ``url_allow_list`` to restrict outbound requests to known, trusted endpoints. + * ``url_allow_list`` is required only when placeholders appear in the destination part of the URL (scheme, host, or port). Placeholders limited to the path or query do not trigger this requirement. + * Leverage ``allow_insecure_http=False`` (default), and consider setting ``allow_credentials=False`` if not needed. + * - Excessive privileges (for :ref:`ServerTool `) + - Run in least-privilege containers/pods. + + Separate network namespaces for sensitive data/external system access. + + Explicitly deny unnecessary filesystem/network access. + * - Stateful tools + - Prefer stateless tools. If stateful, use hardened datastores (e.g., Oracle Database) over in-memory objects where feasible. + + Implement optimistic locking and rigorous input sanitization for state-modifying operations. + * - :ref:`ClientTool ` misuse (client-side vulnerabilities) + - Client apps handling :ref:`ToolRequest ` must treat ``args`` as untrusted. + + Validate/sanitize client-side `args` before local execution (esp. OS commands, sensitive API calls). + * - Insecure underlying components (for tools from Flows, Steps) + - Ensure source Flows or Steps tools for WayFlow :ref:`ServerTool ` are vetted; the tool inherits their security. + * - Data leakage via :ref:`ToolResult ` + - Define ``output_descriptors`` clearly. + + Ensure :ref:`ServerTool ` ``func`` and :ref:`ClientTool ` client code return only necessary data. + + Consuming Agent/Flow must validate/sanitize :ref:`ToolResult ` `content`. + + When sending tool output back to an LLM, prefer a structured encoding and keep it visibly labelled as tool data. + + +Considerations regarding network communication +---------------------------------------------- + +WayFlow components use the network for LLM communications, :ref:`ApiCallStep ` tool operations, and third-party integrations. Implement robust network security. + +.. important:: + + Adopt a defense-in-depth network security strategy for WayFlow, including supply-chain security for models/containers, network segmentation, encrypted communications, and strict egress controls. + +Supply-Chain Security for WayFlow Assets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* **Model Integrity**: For third-party models used by WayFlow (fine-tuned, embeddings): + * Download via HTTPS, pin SHA-256 digests and verify integrity before loading into WayFlow. +* **Container Security**: For containerized WayFlow: + * Use minimal, regularly updated base images. Scan images (e.g., Trivy, Grype) for CVE. Sign (e.g., before pushing to a package index) and verify (e.g., when downloading) images. + +Network Segmentation for WayFlow +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Isolate WayFlow components to limit blast radius: + +* **Subnet Isolation**: + * LLM hosts (OCI Gen AI, vLLM) in dedicated subnets. + * Tool backends and :ref:`ApiCallStep ` targets in isolated subnets. + * Keep the WayFlow *control-flow* logic (e.g., Agents, Flows, BranchingSteps) separate from data processing (e.g., database mutations, data analytics) +* **TLS**: + * Use mTLS for WayFlow service-to-service communications such as connecting to MCP servers, Monitoring/Telemetry services, File Storage Services, Databases, and so on. + +Egress Controls for WayFlow Components +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* **Strict Egress Rules**: Default-deny outbound traffic: + * Allow only Allow-Listed domains/IPs for WayFlow components (such as when using Tools like :ref:`RemoteTool ` with its ``url_allow_list`` parameter, :ref:`ApiCallStep `, or :ref:`PromptExecutionStep `) +* **Centralized Allow-Lists for WayFlow**: + * LLM provider endpoints and Telemetry/logging destinations. + * Approved Tool API endpoints for tools making external calls, such as :ref:`ApiCallStep ` and :ref:`RemoteTool ` (enforced via its ``url_allow_list`` parameter). + * Make sure to set up alert on policy violations. + +Network Connection Requirements +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: WayFlow Network Security: Key Implementations + :widths: 40 60 + :header-rows: 1 + + * - Area + - WayFlow-Specific Implementation + * - External LLM Traffic Encryption + - • OCI Gen AI: HTTPS by default. + + • vLLM: Front with TLS terminator (Nginx/Caddy) or use mTLS ingress. + + • Verify TLS certs; pin where feasible. + * - WayFlow Component Egress Rules + - • Container network policies (iptables, Calico, Cilium) allowing only approved LLM endpoints, Tool API destinations, telemetry sinks, and DNS. + * - External API Call Security (e.g., :ref:`ApiCallStep `, :ref:`RemoteTool `) + - • Enforce HTTPS: Use ``allow_insecure_http=False`` (default for both). + • **URL Allow Lists**: Strongly prefer configuring ``url_allow_list`` for :ref:`RemoteTool ` and other outbound request components. In code, ``url_allow_list`` is required only when an :ref:`ApiCallStep ` or :ref:`RemoteTool ` URL template contains placeholders in the destination part of the URL (scheme, host, or port). Placeholders limited to the path or query do not trigger this requirement. + • Validate/sanitize templated URLs and other request parameters (headers, body) derived from potentially untrusted inputs, and prefer fixed developer-controlled base URLs with templated path/query/body values only. + • Treat warnings for loopback, link-local, or private IP literal targets as a security review signal, not as something to ignore. + • Consider ``allow_credentials=False`` and ``allow_fragments=False`` for :ref:`RemoteTool ` if those features are not strictly necessary. + • Use connection timeouts/rate limiting (tool-specific or via infrastructure). + • Log outbound requests (destination URLs, sanitized headers/bodies) for audit. + * - Data Residency + - • Ensure LLM providers meet data residency needs. + • Use regional LLM endpoints for data locality with WayFlow. + * - Network Monitoring & Alerting + - • Maintain allow-list of approved hostnames and IPs for WayFlow traffic. + + • Alert on policy violations/unauthorized connection attempts from WayFlow components. + + +Considerations regarding API keys and secrets management +-------------------------------------------------------- + +Secure API key/secret management is critical for WayFlow deployments (LLM providers, external services, internal auth) to prevent unauthorized access and data breaches. + +.. important:: + + Never embed API keys, passwords, or other secrets directly in code, configuration files, or serialized JSON/YAML. Always use secure injection mechanisms and follow the principle of least privilege for credential access. + +Control Secret Sprawl +~~~~~~~~~~~~~~~~~~~~~ + +Prevent hardcoded secrets in WayFlow: + +* **Runtime injection for WayFlow required secrets (e.g., LLM API key or database credentials)**: + + * Avoid using Kubernetes/Docker secrets to deploy environment variables. + * Use secrets managers (e.g., OCI Vault). +* **Avoid Baked Secrets in WayFlow**: + + * Never put API keys in WayFlow's serialized JSON/YAML. + * Exclude secrets from Python source shown to LLMs during WayFlow development. + * Use placeholders in version-controlled WayFlow configs. + * Use pre-commit hooks to prevent committing secrets to source control. +* **Per-Flow Service Accounts & Rotation**: + + * Use dedicated service accounts per WayFlow Flow. + * Rotate API keys regularly. + * Automate key rotation if possible. + * Monitor/audit credential use. + +LLM Provider Authentication Security +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**OpenAI Models (`OpenAIModel`):** + +* Injecting a Secret as an environment variable makes the value available to everything inside the Pod. +* We recommend exploring alternatives to sourcing the ``OPENAI_API_KEY`` env variable at runtime. +* Use organization-specific API keys with usage limits. +* Monitor :ref:`TokenUsage ` for anomaly detection. +* Secure proxy credentials separately for VPN proxies. + +**OCI GenAI Models (`OCIGenAIModel`):** + +* **Instance Principal** (:ref:`OCIClientConfigWithInstancePrincipal `): Preferred for OCI Compute (no stored credentials). +* **API Key** (:ref:`OCIClientConfigWithApiKey `): Store private keys in key management; protect ``~/.oci/config`` (permissions 600). +* **User Authentication** (:ref:`OCIClientConfigWithUserAuthentication `): Never log ``key_content``. + +**vLLM Models (`VllmModel`):** + +* Use internal DNS names and private networking +* Implement authentication at reverse proxy level + + +Credential Rotation and Monitoring +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* **Rotation**: Rotate WayFlow credentials regularly. +* **Monitoring**: Track WayFlow's :ref:`TokenUsage ` for anomalies. +* **Incident Response**: Have procedures to revoke compromised WayFlow credentials. + +Considerations regarding Resource-exhaustion vectors +---------------------------------------------------- + +Resource-exhaustion events may stem from hostile over-use or from innocent implementation mistakes (e.g., unbounded loops, forgotten awaits), so interrupts should be viewed as guardrails against both abuse and developer error. +WayFlow provides two **soft execution-interrupts**, but they don't cover all vectors. This section details their use, limitations, and additional production hardening techniques. + +Built-in execution interrupts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 20 80 + + * - Class + - Purpose / behaviour + * - :ref:`SoftTimeoutExecutionInterrupt ` + - Stops the assistant **after** a configurable wall-clock duration (default 10 min). + + • *Soft* → it *waits* until the current Step or FlowExecutionStep returns; it **cannot** pre-empt a long-running Tool call or LLM generation. + * - :ref:`SoftTokenLimitExecutionInterrupt ` + - Aborts execution once an LLM (or group of LLMs) has emitted a specified number of tokens. + + • *Soft* → the check happens between steps; generation already in flight finishes first. + +Usage example +~~~~~~~~~~~~~ + +.. code-block:: python + + from wayflowcore.agent import Agent + from wayflowcore.executors.interrupts.timeoutexecutioninterrupt import SoftTimeoutExecutionInterrupt + from wayflowcore.executors.interrupts.tokenlimitexecutioninterrupt import SoftTokenLimitExecutionInterrupt + + timeout_int = SoftTimeoutExecutionInterrupt(timeout=30) # 30 s max + token_int = SoftTokenLimitExecutionInterrupt(total_tokens=500) # 500 tokens + status = conversation.execute(execution_interrupts=[timeout_int, token_int]) + + +Gaps in coverage (what is **not** provided) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* **Memory limits** – no built-in interrupt for resident-set or GPU VRAM. +* **CPU / thread quotas** – Python loops can still hog a core until the Step returns. +* **Hard timeouts** inside a Tool – a Tool that calls a REST API may block indefinitely. +* **Concurrent-request ceilings** – ``MapStep(parallel_execution=True)`` fans-out one sub-flow (or Tool call) per element in the input list. Because there is no internal throttle, nothing stops 1000 parallel MapStep forks (possibly leading to CPU thrashing or OOM-kill) without your own semaphore. +* **LLM generation cancellation** – once a prompt is sent, the soft timeout waits for the model to answer. + +Recommended hardening layers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. rubric:: OS / Container level + +* **Resource Limits (cgroups/Kubernetes)** for WayFlow containers: + Set CPU/memory requests and limits to prevent resource starvation by WayFlow processes. + Example: + :: + + resources: + requests: { cpu: "500m", memory: "512Mi" } + limits: { cpu: "1", memory: "1Gi" } + +.. rubric:: Application level + +* **Per-Tool timeouts** + + For WayFlow Tools involving long-running I/O, wrap calls in ``concurrent.futures.wait`` or ``subprocess.run`` with a ``timeout``. Raise a retryable error for the Flow. + +* **Concurrency guard** + + For WayFlow's :ref:`MapStep ` or similar fan-out patterns: + + .. code-block:: python + + from asyncio import Semaphore + # Limit concurrent operations in a MapStep + guard = Semaphore(value=16) + +* **Input size validation** + + Limit list lengths, string sizes, and recursion depth for data fed into WayFlow's :ref:`MapStep ` or recursive Agents to prevent exponential resource use. + +* **Circuit-breaker patterns** + + For WayFlow interactions with LLMs or Tools, implement circuit breakers to trip after *n* failures/SLA breaches, returning a graceful error. Consider following code block for conceptual direction: + + .. code-block:: python + + from pybreaker import CircuitBreaker + from wayflowcore.tools import tool, ToolExecutionStep + from requests import get, RequestException + + # ❶ create a breaker: 5 failures trip; reset after 60 s + api_breaker = CircuitBreaker(fail_max=5, reset_timeout=60) + + @tool + def robust_get(url: str) -> str: + """GET url with breaker; raises CircuitBreakerError if OPEN""" + try: + with api_breaker: + resp = get(url, timeout=3) + resp.raise_for_status() + return resp.text + except RequestException as e: + # will count as a failure inside the breaker + raise RuntimeError(f"downstream error: {e}") from e + + step = ToolExecutionStep(robust_get) # use in a Flow + +.. rubric:: LLM usage governance + +* Combine WayFlow's ``SoftTokenLimitExecutionInterrupt`` with server-side LLM provider usage quotas (e.g., OpenAI hard limits, vLLM rate-limiting). + + +Considerations regarding input validation and sanitation +-------------------------------------------------------- + +WayFlow has no built-in input validation/sanitation. Developers are responsible for securing user inputs via validation, sanitation, and external guardrails. + +.. important:: + + All user-provided inputs—whether from chat interfaces, API payloads, or Tool arguments—should be treated as potentially malicious and require rigorous validation and sanitation before processing. + +Core Input Security Requirements +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Validation and Bounds Checking:** + +* Validate length/structure of all inputs to WayFlow (chat, API, Tool arguments). +* Enforce max length limits on user inputs to WayFlow. +* Validate formats/types before passing to WayFlow components. +* Check inputs are within reasonable bounds for the WayFlow application. +* Use ``output_descriptors`` on :ref:`PromptExecutionStep ` for JSON schemas to reduce prompt-injection leakage. + + +**External Guardrails Integration:** + +WayFlow applications should leverage external security services to provide comprehensive input protection: + +* **OCI Gen AI Inference Protection**: Built-in guardrails (toxicity, prompt-injection, jailbreak scanning) per Responsible AI. +* **Custom filtering for WayFlow**: Use regex/policy filters with open-source models in WayFlow. +* **LLM DevSecOps**: Apply `LLM DevSecOps guide `_ practices to WayFlow development. + +**Architectural Patterns for Input Security:** + +* **Early filtering**: Insert a :ref:`BranchingStep ` at Flow start to route "unsafe" content to a refuse/retry branch. +* **Defense in depth**: Layer multiple validation/sanitation for WayFlow inputs. +* **Fail-safe defaults**: Reject ambiguous/malformed inputs to WayFlow. + +Considerations regarding logging +-------------------------------- + +Error handling & exception hygiene +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Robust error handling is key for WayFlow stability and security. + +* **Custom WayFlow Steps**: Implement comprehensive error handling. Map exceptions to sanitized messages, avoiding internal detail leakage. +* **`CatchExceptionStep`**: Configure to prevent stack trace/sensitive debug info leakage in user errors. +* **Private Debug Logging**: Log full exception traces for WayFlow errors privately for debugging, while sanitizing user-facing messages. + +Audit & observability +~~~~~~~~~~~~~~~~~~~~~ + +Comprehensive auditing/observability is vital for WayFlow. + +* **Structured Logs for WayFlow Events**: Log key WayFlow operations at ``INFO`` level: + + * ``conversation_id``: Trace interaction lifecycle. + * ``step_name``: Executed WayFlow Step. + * ``input_hash``: Hash of Step/Tool input (avoid raw sensitive input). + * ``output_hash``: Hash of Step/Tool output. + * ``tool_name``: Called Tool. +* **Tamper-Evident Log Storage for WayFlow**: Use secure, append-only storage for WayFlow logs. +* **PII Scrubbing in WayFlow Logs**: Mask/redact/tokenize PII in WayFlow logs/traces before persistence for privacy compliance (GDPR, right to be forgotten). It may be beneficial to use frameworks for detecting, redacting, masking, and anonymizing sensitive data (PII) such as `Presidio `_ + +Considerations regarding Telemetry & observability +-------------------------------------------------- + +WayFlow can emit rich execution traces—**Spans**—that describe every +tool call, LLM invocation, and Step transition inside an Agent or Flow. +These traces are invaluable for debugging and performance tuning, **but +they may also contain sensitive data**. + +1. What's in a Span? +~~~~~~~~~~~~~~~~~~~~~ + +* Flow / Agent configuration snapshot +* Sequence of executed Steps (timestamps, duration) +* Input / output payloads, including user prompts and Tool arguments +* Status transitions and exception details + +2. Default behaviour +~~~~~~~~~~~~~~~~~~~~~ + +WayFlow sends **no traces by default**. +To enable exporting you must supply a custom +:ref:`SpanExporter `. + +.. danger:: + Raw Spans can expose PII, secrets, or proprietary model prompts. PII/secrets must be removed/hashed from logs/metrics. + +3. Implementing a SpanExporter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: python + + from wayflowcore.tracing.spanexporter import SpanExporter + from wayflowcore.tracing.span import Span + import asyncio, aiohttp + + class AsyncHTTPSExporter(SpanExporter): + async def _send(self, span: Span) -> None: + payload = sanitize(span) # e.g., redact secrets here + async with aiohttp.ClientSession() as s: + await s.post("https://trace.local/ingest", json=payload, timeout=3) + + def export(self, span: Span) -> None: + asyncio.create_task(self._send(span)) # non-blocking + +Key points: + +* **Async / non-blocking** - keep the exporter off the critical path. +* **Robust error handling** - never raise from ``export``; drop or queue on + failure. +* **Back-pressure** - apply rate limits or batch Spans to avoid DoS on the + collector. + +4. SpanProcessor guidance +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When you add a custom :ref:`SpanProcessor `: + +* Apply sampling early to reduce volume. +* Use bounded queues to cap memory. +* Log (INFO) when Spans are dropped due to back-pressure. + + +Considerations regarding UI rendering +------------------------------------- + +WayFlow uses **Jinja 2** for templates in core components like: + +* :ref:`TemplateRenderingStep ` +* :ref:`OutputMessageStep ` +* :ref:`ContextProvider `-driven templates interpolating values into messages + +Jinja 2's automatic HTML-escaping is **disabled by design** in WayFlow to: + +* deliver prompt text to LLMs without foreign escape sequences, and +* prevent mangling of deliberately model-generated HTML tags. + +This preserves model fidelity/latency but removes default XSS/code-injection safeguards. + +WayFlow relies on a stricter implementation of the Jinja's SandboxedEnvironment for security reasons. +Every callable is considered unsafe, and every attribute access is prevented, except for: + +* The attributes ``index0``, ``index``, ``first``, ``last``, ``length`` of the ``jinja2.runtime.LoopContext``; +* The entries of a python dictionary (only native type is accepted); +* The items of a python list (only native type is accepted). + +You should never write a template that includes a function call, or access to any internal attribute or element of +an arbitrary variable: that is considered unsafe, and it will raise a ``SecurityException``. + +Moreover, WayFlow performs additional checks are performed on the inputs provided for rendering. +In particular, only elements and sub-elements that are of basic python types +(``str``, ``int``, ``float``, ``bool``, ``list``, ``dict``, ``tuple``, ``set``, ``NoneType``) +are accepted. In any other case, a ``SecurityException`` is raised. + +.. important:: + + **Never render the raw output of a WayFlow step directly to a browser.** + Treat every ``step.`` or ``Message.content`` string as *untrusted* + and sanitise or escape it before display. Failure to do so can lead to + HTML/JS injection (stored XSS) where malicious scripts sent in chat can be + executed by the browser if auto-escaping is off. + +Recommended mitigations +~~~~~~~~~~~~~~~~~~~~~~~ + +#. **Segregate rendering responsibilities** + + * Use WayFlow only for *data generation*. + * Delegate structured text generation (e.g., HTML, XML, Markdown, SQL) to a + dedicated renderer that offers context-aware auto-escaping. + +#. **Sanitise any LLM or user-supplied markup** + + For example, to sanitise HTML: + + .. code-block:: python + + import bleach + + safe_html = bleach.clean( + raw_wayflow_output, + tags=["b", "i", "u", "br", "p"], # whitelisted tags + attributes={}, + strip=True + ) + + Similar libraries exist for other markup languages. + +#. **Adopt strict content policies** + + Where the rendering context provides it (e.g., web browsers), enforce a + strict Content-Security-Policy to mitigate the risk of unsanitised content + executing. For other contexts (e.g., generating code), ensure the output is + treated as data and never directly executed. + +#. **Use an “escape hatch” for deliberate rich content** + + If you *need* the LLM to emit rich content (e.g., HTML e-mails, reports), + treat it as a download-only artifact where possible. If it must be + displayed, use a sandboxed environment (like an ``