This repository contains the material for the SAP TechEd 2025 session called AI161 - Hands-on: Benchmark, evaluate, and optimize prompts across models.
This session introduces attendees to our new AI Core services for prompt optimizations and evaluation.
Get hands-on experience with the prompt editor, registry, and benchmarking tools available from the generative AI hub capability in SAP AI Core. You can compare models, evaluate performance, and optimize prompts. Learn how to build resilient, model-agnostic AI workflows and avoid vendor lock-in.
The exercise is designed to be done in the SAP AI Launchpad. For the Hands-on session it is designed for we have provided an SAP AI Launchpad and a technical user to login. If you want to repeat this tutorial outside of the hands-on session you will require your own AI Launchpad instance.
Let's get started with the exercises!
- Getting Started
- Exercise 1 - Setup
- Exercise 2 - Evaluate the original prompt
- Exercise 3 - Optimize the original prompt
- Exercise 4 - Evaluate the optimized prompt
IMPORTANT
Please read the CONTRIBUTING.md to understand the contribution guidelines.
Please read the SAP Open Source Code of Conduct.
Support for the content in this repository is available during the actual time of the online session for which this content has been designed. Otherwise, you may request support via the Issues tab.
Copyright (c) 2025 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0 except as noted otherwise in the LICENSE file.