Skip to content

Add Trustabl security scanning to CI - #182

Open
joshua-trustabl wants to merge 1 commit into
joinly-ai:mainfrom
joshua-trustabl:add-trustabl-action
Open

Add Trustabl security scanning to CI#182
joshua-trustabl wants to merge 1 commit into
joinly-ai:mainfrom
joshua-trustabl:add-trustabl-action

Conversation

@joshua-trustabl

Copy link
Copy Markdown

We came across your repo and we like how you are creating a framework to bridge the gap between human interactions and AI capabilities in video conferencing. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.

  1. [MEDIUM] LangChain tool returns its output directly, bypassing the model
    File: examples/client_example.py
    What it means: This tool sets return_direct=True, so LangChain returns the tool's raw output straight to the caller and stops the agent loop — the model never sees the result, cannot validate or summarize it, and any output guardrail or post-processing step is skipped.

  2. [MEDIUM] Mutating tool has no idempotency key
    File: joinly/server.py
    What it means: The tool name signals a side effect (create/send/refund/…), but there's no way to ensure repeated invocations don't lead to unintended duplicate actions if the context isn't perfectly managed.

Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.

Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant