Skip to content

edenciso/agent-sprawl-optimizer

BranchesTags

Repository files navigation

ValueOS Agent Sprawl Optimizer — AI Stack Value Assessment (Beta)

AWS SAM Serverless deployment implementing the non-telemetry (JSON intake) workflow of an Agentic AI Stack Value Assessment.
Forward Deployed Engineers (FDEs) spin up an isolated stack per client, upload structured intake files, and receive a comprehensive AI-generated Agent Sprawl report.

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                     FDE / Client Browser                    │
└──────────────────────────┬──────────────────────────────────┘
                           │  HTTPS
                           ▼
              ┌───────────────────────┐
              │   Amazon API Gateway  │◄──── Cognito Authorizer
              │   (REST, /beta)       │
              └───────┬───────────────┘
                      │
        ┌─────────────┼─────────────────────┐
        ▼             ▼                     ▼
  ┌──────────┐  ┌────────────┐     ┌──────────────┐
  │Engagements│  │Intake Upload│    │ Orchestrator  │
  │  Lambda   │  │  Lambda    │     │   Lambda      │
  └─────┬─────┘  └─────┬──────┘    └──────┬────────┘
        │               │                  │
        ▼               ▼                  ▼
  ┌──────────┐   ┌───────────┐   ┌─────────────────────────┐
  │ DynamoDB  │   │  S3 Intake│   │   Step Functions        │
  │ (3 tables)│   │   Bucket  │   │   Audit Workflow        │
  └───────────┘   └───────────┘   │                         │
                                  │  1. Intake Processor     │
                                  │  2. ┌─Scoring Engine     │
                                  │     ├─Redundancy Detect. │
                                  │     └─Savings Modeler    │
                                  │  3. Report Generator     │
                                  └────────────┬────────────┘
                                               │
                        ┌──────────────────────┼──────────┐
                        ▼                      ▼          ▼
                  ┌──────────┐          ┌──────────┐  ┌───────┐
                  │  Bedrock  │          │ DynamoDB │  │  S3   │
                  │  Claude   │          │ (results)│  │Reports│
                  └──────────┘          └──────────┘  └───────┘

Agent Fleet

The agent sprawl audit pipeline runs five specialized Claude agents orchestrated by AWS Step Functions.

Agent Bedrock Model Purpose
Intake Processor Claude Sonnet 4.5 Validates, normalizes, and deduplicates the three intake JSON files into a canonical tool inventory
Scoring Engine Claude Sonnet 4.5 Scores each tool on utility, integration, and cost-effectiveness (1–10 each); assigns KEEP / EVALUATE / CONSOLIDATE / ELIMINATE actions
Redundancy Detector Claude Sonnet 4.5 Identifies overlapping capability clusters, selects survivors, flags zombie tools (<20% utilization, >$100/mo)
Savings Modeler Claude Sonnet 4.5 Projects conservative / moderate / aggressive savings scenarios with 12-month cash-flow models
Report Generator Claude Sonnet 4.5 Synthesizes a 2,000–3,000-word Markdown audit report with executive summary, scoring matrix, redundancy map, savings forecast, and prioritized roadmap

All five agents use the US cross-region inference profile (us.anthropic.claude-sonnet-4-5-20250929-v1:0), which automatically routes requests across us-east-1, us-east-2, and us-west-2 for higher throughput and availability at no additional charge.

Prerequisites

Before deploying, ensure the following tools and permissions are available on your workstation.

Requirement Version Notes
AWS CLI ≥ 2.x Configured with credentials (aws configure)
AWS SAM CLI ≥ 1.100 Install guide
Python 3.12 For Lambda runtime compatibility
Docker latest SAM uses Docker for sam build
Bedrock Model Access — Complete the one-time Anthropic use-case form in the Bedrock console. The US inference profile us.anthropic.claude-sonnet-4-5-20250929-v1:0 must show as Active in your region.

Bedrock Model Access — Important

SAM will deploy successfully without model access, but the audit pipeline will fail at runtime. Go to Amazon Bedrock → Model access and complete the one-time Anthropic use-case form. Then verify the US inference profile is active:

aws bedrock list-inference-profiles --region us-east-1 \
  --query 'inferenceProfileSummaries[?inferenceProfileId==`us.anthropic.claude-sonnet-4-5-20250929-v1:0`].{Name:inferenceProfileName,Status:status}' \
  --output table

Project Structure

sprawl-optimizer-sam/
├── template.yaml                  # SAM infrastructure-as-code (all resources)
├── samconfig.toml                 # Default deployment parameters
├── Makefile                       # Build / deploy / test shortcuts
├── README.md                      # This file
│
├── statemachine/
│   └── audit_workflow.asl.json    # Step Functions state machine definition
│
├── schemas/                       # JSON intake file schemas (for client reference)
│   ├── intake_schema_1_org_profile.json
│   ├── intake_schema_2_tool_catalog.json
│   └── intake_schema_3_cost_data.json
│
├── layers/
│   └── common/
│       └── python/
│           ├── requirements.txt   # Layer dependencies (boto3, jsonschema)
│           └── shared/            # Shared library consumed by all Lambdas
│               ├── __init__.py
│               ├── bedrock_client.py   # Claude invocation wrappers
│               ├── dynamo_client.py    # DynamoDB CRUD helpers
│               ├── s3_client.py        # S3 upload/download/presign helpers
│               ├── prompts.py          # Agent system prompts
│               └── models.py           # JSON Schema definitions for validation
│
├── functions/                     # One directory per Lambda function
│   ├── api_engagements/           # CRUD for engagement records
│   ├── intake_upload/             # Upload & validate intake JSONs
│   ├── intake_processor/          # Agent: normalize intake → canonical inventory
│   ├── scoring_engine/            # Agent: score tools on 3 dimensions
│   ├── redundancy_detector/       # Agent: cluster overlapping tools
│   ├── savings_modeler/           # Agent: project 3-scenario savings
│   ├── report_generator/          # Agent: synthesize Markdown audit report
│   └── orchestrator/              # Start pipeline, check status, retrieve report
│
└── tests/
    └── events/                    # Sample intake files for testing
        ├── sample_org_profile.json
        ├── sample_tool_catalog_sales.json
        └── sample_cost_data.json

Deployment

Quick Start (Guided)

The guided deploy walks you through every parameter interactively. This is the recommended path for first-time setup.

# 1. Build all Lambda functions and the shared layer
make build

# 2. Deploy with interactive prompts
make deploy-guided

SAM will prompt you for:

Parameter What to Enter Example
Stack Name Unique CloudFormation stack name sprawl-audit-techscale
AWS Region Region with Bedrock access us-east-1
ClientName Slug identifying the client (lowercase, no spaces) techscale
AdminEmail FDE's email (receives Cognito temp password) fde@valuelayer.com
StageName API stage beta

After deployment completes (~3 minutes), SAM prints the stack outputs. Save these — you'll need them for API calls.

Quick Start (Non-Interactive)

For repeat deployments or CI/CD pipelines, use the samconfig.toml defaults.

# Edit samconfig.toml with your parameters first, then:
make build
make deploy

What Gets Created

The CloudFormation stack provisions the following resources (all names are prefixed with your ClientName):

  • Cognito User Pool with a pre-created admin user (temp password sent to AdminEmail)
  • API Gateway REST API at /beta stage with Cognito authorization
  • 3 DynamoDB tables (Engagements, ToolInventory, AnalysisResults) — pay-per-request billing
  • 2 S3 buckets (intake files, generated reports) — encrypted at rest, no public access
  • 8 Lambda functions on ARM64 Graviton with shared layer
  • Step Functions state machine orchestrating the 4-stage audit pipeline

API Reference

All endpoints require a Cognito JWT in the Authorization header. To obtain a token, authenticate against the Cognito User Pool using the AWS CLI or any OIDC-compatible client.

Authentication

# Get your initial token (first login requires NEW_PASSWORD_REQUIRED challenge)
aws cognito-idp initiate-auth \
  --client-id <CognitoClientId from outputs> \
  --auth-flow USER_PASSWORD_AUTH \
  --auth-parameters USERNAME=admin,PASSWORD=<temp-password-from-email>

# After setting a new password, subsequent logins return an IdToken directly.
# Use the IdToken as: Authorization: Bearer <IdToken>

Engagements

POST /engagements — Create a new client engagement.

{
  "client_name": "TechScale Solutions",
  "client_contact": "jennifer.martinez@techscale.com",
  "fde_name": "Alex Kim",
  "notes": "85-person SaaS company, expects 30% savings target"
}

GET /engagements — List all engagements for this stack.

GET /engagements/{id} — Get full engagement details including status and savings summary.

Intake Upload

POST /engagements/{id}/intake — Upload an intake file. The request body is the raw JSON content of one of the three intake types. Set the X-Intake-Type header to one of: org_profile, tool_catalog, or cost_data.

curl -X POST "$API_URL/engagements/$ENG_ID/intake" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Intake-Type: org_profile" \
  -d @tests/events/sample_org_profile.json

GET /engagements/{id}/intake — Check which intake files have been uploaded.

Orchestration

POST /engagements/{id}/run — Start the audit pipeline. Requires at least one intake file uploaded. Returns the Step Functions execution ARN.

GET /engagements/{id}/status — Poll pipeline status. Returns engagement status (intake, processing, complete, failed) and Step Functions execution state.

GET /engagements/{id}/report — Retrieve presigned download URLs for the generated audit report (Markdown) and raw analysis data (JSON). URLs expire after 1 hour.

End-to-End Workflow

This is the complete FDE workflow from stack deployment to delivering a client report.

1. Deploy the Stack

make build && make deploy-guided

2. Authenticate

Retrieve the temporary password from email, authenticate with Cognito, and set a permanent password.

3. Create an Engagement

curl -X POST "$API_URL/engagements" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"client_name":"TechScale Solutions","fde_name":"Alex Kim"}'

Save the returned engagement_id.

4. Upload Intake Files

Work with the client to fill out the three intake JSON files (schemas are in schemas/). Upload each one.

for TYPE in org_profile tool_catalog cost_data; do
  curl -X POST "$API_URL/engagements/$ENG_ID/intake" \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -H "X-Intake-Type: $TYPE" \
    -d @client_data/${TYPE}.json
done

5. Run the Audit

curl -X POST "$API_URL/engagements/$ENG_ID/run" \
  -H "Authorization: Bearer $TOKEN"

6. Monitor Progress

# Poll until status is "complete" (typically 3-8 minutes)
watch -n 15 "curl -s '$API_URL/engagements/$ENG_ID/status' \
  -H 'Authorization: Bearer $TOKEN' | python3 -m json.tool"

7. Download the Report

curl -s "$API_URL/engagements/$ENG_ID/report" \
  -H "Authorization: Bearer $TOKEN" | python3 -m json.tool

# Use the presigned URLs to download report.md and audit_data.json

Testing with Sample Data

The tests/events/ directory contains realistic sample intake files for a fictional 85-person SaaS company called TechScale Solutions. These files demonstrate the expected structure including edge cases like zombie tools, redundant subscriptions, and shadow AI.

# Quick test using the sample data
make test-upload ENG_ID=<your-engagement-id> TOKEN=<your-token>

Or upload them manually following the workflow above, substituting tests/events/sample_org_profile.json, tests/events/sample_tool_catalog_sales.json, and tests/events/sample_cost_data.json for the client data files.

Intake File Schemas

Clients (or FDEs on their behalf) prepare three JSON files before the audit. Full JSON Schema definitions are in the schemas/ directory. Here is a summary of what each file captures.

1. Organization Profile (org_profile)

Company metadata and departmental structure. Includes company name, size, industry, departments with headcount, designated leads, and any known pain points around their AI tooling.

2. Tool Catalog (tool_catalog)

Department-by-department inventory of AI tools currently in use (to the best of the client's knowledge). Each entry includes the tool name, vendor, primary use cases, estimated frequency of use, and any known integration points. This does not need to be exhaustive — the Intake Processor agent cross-references against the cost data to surface tools that appear in billing but not in the catalog (shadow AI signal).

3. Cost Data (cost_data)

Finance-sourced subscription and billing data. Each line item includes the tool/vendor name, monthly cost, billing cycle, contract dates, number of seats, and estimated utilization percentage. This file also captures budget context (current total spend, target savings) and any known untracked expenses.

Customization

Adjusting Agent Behavior

All agent system prompts live in layers/common/python/shared/prompts.py. You can tune scoring thresholds, redundancy sensitivity, savings scenario assumptions, and report structure by editing these prompts and redeploying.

# After editing prompts.py:
make build && make deploy

Adding More Intake Types

The architecture is designed to accommodate additional intake types (e.g., SSO export data, API gateway logs) as the product matures toward the full telemetry path. To add a new type, you would add its JSON Schema to models.py, update the intake upload validation logic, and extend the Intake Processor prompt to handle the new data source.

Switching Bedrock Models

To use a different Claude model (e.g., Opus for deeper reasoning, or Haiku for lower cost), update the inference profile ID in template.yaml:

# In Globals > Function > Environment > Variables, change:
CLAUDE_MODEL_ID: "us.anthropic.claude-sonnet-4-5-20250929-v1:0"
# To another inference profile, e.g.:
CLAUDE_MODEL_ID: "us.anthropic.claude-haiku-4-5-20251001-v1:0"

Then update the MAX_OUTPUT_TOKENS constant in bedrock_client.py to match the new model's ceiling. List available profiles with:

aws bedrock list-inference-profiles --region us-east-1 \
  --query 'inferenceProfileSummaries[?contains(inferenceProfileId, `anthropic`)].{ID:inferenceProfileId, Name:inferenceProfileName, Status:status}' \
  --output table

Switching AWS Regions

Bedrock inference profile availability varies by region. The default configuration targets us-east-1 with the US cross-region inference profile. If deploying to another region, verify that your chosen profile is available with aws bedrock list-inference-profiles, then update samconfig.toml.

Teardown

To remove all resources created by a stack (including the DynamoDB data and S3 objects):

# Empty the S3 buckets first (CloudFormation can't delete non-empty buckets)
make clean-buckets STACK=sprawl-audit-techscale

# Then delete the stack
make destroy STACK=sprawl-audit-techscale

Cost Estimate

For a typical single-client audit (one pipeline run with ~15 tools):

Resource Estimated Cost
Bedrock Claude Sonnet 4.5 (5 agents, 5 calls) $0.15 – $0.60 per run
Lambda compute (8 functions, ~5 min total) < $0.01
Step Functions (1 execution, ~10 transitions) < $0.01
DynamoDB (on-demand, ~50 read/write units) < $0.01
S3 storage (intake + reports) < $0.01
API Gateway (< 20 requests per audit) < $0.01
Cognito (1 user) Free tier
Total per audit run ~$0.15 – $0.60

The dominant cost is Bedrock inference. Larger tool inventories (30+ tools) with richer context push toward the higher end. Infrastructure costs are negligible at beta-stage volumes. Cross-region inference pricing is the same as standard regional pricing for geo-specific profiles.

Roadmap to Production

This beta implements the non-telemetry JSON intake path only, using Claude Sonnet 4.5 via cross-region inference. The production roadmap includes:

  1. MCP Server Integration — SSO, billing, and network telemetry connectors for automated discovery (eliminates manual intake files)
  2. Multi-engagement Dashboard — FDE portal for managing concurrent client audits
  3. Report Templating — Branded PDF/PPTX export using the Report Generator output
  4. Webhook Notifications — Slack/email alerts when pipeline stages complete
  5. Client Self-Service Portal — Secure intake file upload UI for clients (removes FDE from the upload step)

License

Proprietary — ValueLayer. All rights reserved.

About

Agent Sprawl Optimizer - AI Stack Value Assessment. Agentic audit pipeline using AWS Serverless Bedrock and Claude deployment for the non-telemetry JSON intake path

valuelayer.ai/valueos

Topics

aws serverless bedrock claude agentic-ai

Resources

Readme

License

Apache-2.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages