pi-langfuse

English | 简体中文

Langfuse observability extension for Pi Coding Agent. It sends complete Pi runs to Langfuse so the prompt, agent workflow, LLM generations, tool calls, final response, usage, cost, and health scores appear in one trace.

What This Adds to Pi

• One Langfuse trace per user prompt, grouped by Pi session.
• Root agent, per-request generation, and per-tool tool observations.
• Final assistant output capture, tool error visibility, and trace-level scores.
• Privacy controls for inputs, outputs, tool I/O, system prompt, and cwd.
• Secret redaction and local path hashing before upload.
• Capability-gated REST fallback for self-hosted Langfuse setups that expose the legacy trace API when OTel spans arrive but traces do not materialize. Langfuse v4 events_only deployments use OTel without legacy fallback ingestion.

Prerequisites

• Node.js >= 22
• Pi Coding Agent installed and configured
• A Langfuse account (cloud or self-hosted)

Quick Start

1. Install the extension:

   pi install npm:pi-langfuse

2. Run Pi once. If no credentials are configured yet, Pi prompts for:

   • Langfuse public key, starting with pk-lf-...
   • Langfuse secret key, starting with sk-lf-...
   • Langfuse host, defaulting to https://cloud.langfuse.com

3. Run Pi normally:

   pi "Explain the architecture of Redis"

4. Open Langfuse and inspect the new trace.

Configuration

Langfuse API keys are available in Langfuse Cloud -> Settings -> API Keys.

Method 1: Interactive setup

Run any pi command with the extension loaded. On first run without configuration, Pi prompts in the CLI or TUI and saves the result to ~/.pi/agent/pi-langfuse/config.json.

To run setup again:

/langfuse-setup

To inspect the active configuration without exposing secrets:

/langfuse-status

The status command reports the config source, host, masked public key, capture policy, active-run state, config path, and last runtime error.

Method 2: Environment variables

Set these before starting Pi:

export LANGFUSE_PUBLIC_KEY="pk-lf-xxxx"
export LANGFUSE_SECRET_KEY="sk-lf-xxxx"
export LANGFUSE_BASE_URL="https://cloud.langfuse.com"  # optional; LANGFUSE_HOST is also supported

Saved config takes precedence. Environment variables are only used when ~/.pi/agent/pi-langfuse/config.json is missing or incomplete.

For short-lived SDK hosts, set the bounded final score-delivery attempt during shutdown:

export PI_LANGFUSE_SCORE_SHUTDOWN_TIMEOUT=2  # seconds; defaults to 2 seconds

The extension attempts queued trace-level scores before other shutdown telemetry work. This value cannot extend the overall shutdown deadline.

Privacy controls can also be set through environment variables:

export LANGFUSE_PRIVACY_PRESET="full-debug"

Available presets:

Preset	Captures
metadata-only	Metadata only; omits inputs, outputs, tool I/O, system prompt, and cwd
prompts-only	Prompt/provider inputs plus metadata
conversations	Inputs and assistant outputs, but omits tool I/O, system prompt, and cwd
full-debug	Full trace detail; this is the default

Fine-grained flags override presets:

export LANGFUSE_CAPTURE_INPUTS=true
export LANGFUSE_CAPTURE_OUTPUTS=true
export LANGFUSE_CAPTURE_TOOL_IO=false
export LANGFUSE_CAPTURE_SYSTEM_PROMPT=false
export LANGFUSE_CAPTURE_CWD=false
export LANGFUSE_CAPTURE_SOURCE_METADATA=false

Source metadata remains off in every preset unless LANGFUSE_CAPTURE_SOURCE_METADATA=true is set explicitly.

All captured payloads are redacted before upload. The extension masks common API keys, bearer tokens, passwords, cookies, private keys, Langfuse keys, GitHub/npm/AWS-style tokens, and local absolute paths.

Payload limits

Before upload, payloads are shaped: strings are truncated and deeply nested or very wide structures are trimmed. These caps keep traces small and protect the Langfuse ingestion pipeline. Override any of them (no rebuild needed):

export PI_LANGFUSE_MAX_STRING_LENGTH=12000       # per-string chars (system prompt, inputs)
export PI_LANGFUSE_MAX_TOOL_PAYLOAD_LENGTH=24000 # per tool input/output chars
export PI_LANGFUSE_MAX_DEPTH=6                    # max nesting depth
export PI_LANGFUSE_MAX_ARRAY_ITEMS=50            # max array elements kept
export PI_LANGFUSE_MAX_OBJECT_KEYS=80            # max object keys kept
export PI_LANGFUSE_MAX_PAYLOAD_NODES=2000        # max total nodes per payload

Set any limit to 0, off, none, or unlimited to disable that cap entirely (captures the full value). Unset or invalid values fall back to the defaults shown above. To capture a very large system prompt or big tool payloads in full, raise or disable the relevant limit (e.g. PI_LANGFUSE_MAX_STRING_LENGTH=off).

Method 3: Persistent config.json

Create or update ~/.pi/agent/pi-langfuse/config.json:

{
  "publicKey": "pk-lf-xxxx",
  "secretKey": "sk-lf-xxxx",
  "host": "https://cloud.langfuse.com",
  "privacyPreset": "conversations"
}

Fine-grained capture flags can also be persisted:

{
  "publicKey": "pk-lf-xxxx",
  "secretKey": "sk-lf-xxxx",
  "host": "https://cloud.langfuse.com",
  "capture": {
    "LANGFUSE_PRIVACY_PRESET": "metadata-only",
    "LANGFUSE_CAPTURE_INPUTS": "true"
  }
}

Security: Keep ~/.pi/agent/pi-langfuse/config.json private. Never commit API keys to version control. When the extension writes this file itself, it creates the config directory with 0700 permissions and the file with 0600 permissions where the host filesystem supports POSIX modes.

Verify the Extension

Check that Pi has loaded the package:

pi list

pi-langfuse should appear in the installed package list.

To verify the Langfuse host and API keys from inside Pi, run:

/langfuse-test

This command makes a timeout-bounded authenticated request to Langfuse and, if it succeeds, sends a small test trace.

What Appears in Langfuse

• Each Pi session gets its own Langfuse session ID.
• Each user prompt within that session becomes a separate trace.
• The trace contains the final assistant output shown in Pi.
• Tool runs appear as tool observations with arguments, results, and error state.
• LLM requests appear as generation observations, including usage and cost when the provider exposes them.
• Trace-level scores include tool counts, tool success rate, and whether the run had errors.

The package also includes a Langfuse CLI skill, so Langfuse data can be queried directly from Pi:

/pi-langfuse-langfuse <your-query>

Source Metadata

Repository source capture is independent of the privacy presets and disabled by default. Enable it only after deciding that commit identity is appropriate for the Langfuse project:

export LANGFUSE_CAPTURE_SOURCE_METADATA=true

For a Git worktree, the extension records only revision state:

{
  "source_type": "git-repo",
  "vcs.ref.head.revision": "0123456789abcdef...",
  "git_detached": "false",
  "git_dirty": "false",
  "metadata_source": "git-detection"
}

The revision is the full HEAD commit. Dirty state includes tracked changes and untracked files, but never their paths or contents. Detached state is reported without a branch or tag name. Git remotes, URLs, credentials, usernames, branches, absolute paths, and repository names are never inspected or uploaded by this collector.

When capture is off, the collector does not invoke Git and reports source_type: "disabled". Non-Git directories report non-git; a missing or unusable Git executable and incomplete Git state report unavailable.

Use native Langfuse and OpenTelemetry settings for deployment identity and explicit operator-owned overrides instead of repository files:

export LANGFUSE_RELEASE="1.2.3"
export LANGFUSE_TRACING_ENVIRONMENT="production"
export OTEL_SERVICE_NAME="pi-agent"
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.2.3,vcs.repository.name=public-repo"

LANGFUSE_RELEASE and LANGFUSE_TRACING_ENVIRONMENT keep their Langfuse semantics. OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES are loaded as process-scoped OpenTelemetry resource attributes; values such as vcs.repository.name apply to every session sharing the runtime, may identify private source, and require a runtime restart to change.

Compatibility and rollback

Earlier versions emitted git_commit, branch, remote, owner, repository, and repo-local .pi-langfuse.metadata.json values by default. New traces use vcs.ref.head.revision and stop reading that file. Update dashboards before enabling source capture; historical traces are unchanged.

Unset LANGFUSE_CAPTURE_SOURCE_METADATA to stop collection immediately. Pin pi-langfuse@1.5.12 only if the old schema is required during migration; doing so also restores its broader default source disclosure.

Troubleshooting

No traces appearing?

• Verify the API keys and run /langfuse-setup again if needed.
• Run /langfuse-status to confirm the loaded host, config source, privacy mode, and last runtime error.
• Confirm the Langfuse project is active and accepts writes.
• Confirm the keys have write permission.
• Look for 📊 Langfuse: log messages in Pi output.

Extension not loading?

pi list
pi install npm:pi-langfuse

"Missing config" on startup?

• Run /langfuse-setup.
• Or set LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY before starting Pi.

Model or cost not showing?

• Some providers do not expose cost information.
• Inspect the raw observation data in Langfuse traces.
• The model field can come from provider events, finalized assistant messages, model_select, or ctx.model.

API key errors?

• Public keys start with pk-lf-.
• Secret keys start with sk-lf-.
• For self-hosted deployments, verify the host URL.

Development Docs

Development setup, source installation, runtime architecture, trace model, tracked fields, and validation steps are documented in DEVELOPMENT.md and DEVELOPMENT_CN.md.

License

MIT