feat: Lakebase deployment, auth improvements, and PBI model routing#51
Merged
nehmetohmedb merged 212 commits intoJun 4, 2026
Conversation
The token telemetry callback was opening new database sessions via get_auth_context() to look up PAT tokens, which caused session conflicts with ongoing transactions during crew creation. Changes: - Add skip_db_auth parameter to get_auth_context() to skip PAT database lookup - Add skip_db_auth parameter to send_logfood_telemetry() for pass-through - Update LiteLLM telemetry callbacks to use skip_db_auth=True This fixes the 'Could not refresh instance' error when creating crews.
Ensure telemetry during agent execution doesn't open database sessions, preventing potential session conflicts and connection pool issues.
- Fix embedding telemetry using correct product_context (EMBEDDING instead of LLM) - Add console handler for subprocess logging in Databricks Apps (uses sys.__stderr__) - Configure src.utils.telemetry logger in subprocess for embedding telemetry visibility - Add user_token support to send_logfood_telemetry for OBO authentication in subprocesses - Add module-level _subprocess_user_token fallback in llm_manager for callback threads - Remove redundant LiteLLM telemetry wrapper from process_crew_executor (was causing double logging) - Improve telemetry log messages with consistent [LogfoodTelemetry] prefix and structured output
- Add 'secret' context to Databricks Secrets service API calls - Add 'connection_test' context to Databricks connection test calls - Add 'kasal_lakebase' User-Agent for Lakebase operations - Change MLflow User-Agent to 'kasal_mlflow' for better attribution
…ions Centralized User-Agent configuration for consistent telemetry tracking in Databricks logfood tables. Changes: - Added MCP, LAKEBASE, MLFLOW, and SECRET to KasalProduct enum in telemetry.py - Updated MCP adapter to use get_user_agent(KasalProduct.MCP) for kasal_mcp/0.1.0 tracking - Standardized all services to use KasalProduct constants instead of hardcoded strings - Ensured consistent User-Agent format (kasal_<product>/<version>) across: - MCP Adapter - MLflow Service - Lakebase Connection Service - Databricks Secrets Service - Databricks Service (connection test) - Vector Endpoint Repository This enables accurate Kasal usage tracking in Databricks telemetry and prepares for partner integration tracking via workload_insights table.
Addresses code review feedback from MrBlack1995: - Add Tuple to typing imports - Add full type annotations to _should_send method signature
Resolved conflict in main.py: kept colleague's ASGI class-based LocalDevAuthMiddleware, preserving settings.LOCAL_DEV_USER_EMAIL fallback instead of hardcoded admin@admin.com. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ce docs Phase 4 — Runtime output scanning & excessive agency: - Secret leak detection (10 credential pattern families) in agent output - Flow trust boundary scanning between crews in multi-crew flows - Memory poisoning defense (scan task output before persistence) - Tool output scanning in step callbacks - Excessive agency detection (PERFORMS_DESTRUCTIVE_OPERATIONS flag) Phase 5 — Optimizations: - Unified SecurityScannerPipeline singleton with audit logging - False-positive reduction (tightened MEDIUM regex patterns) - LLM guardrail SHA-256 LRU caching (skip redundant calls on retries) - Secret detector expansion (GitHub, GCP, Azure, DSA/encrypted PEM) Documentation: - Updated README_SECURITY_COMPLIANCE.md with Areas 9-16 and overdelivery table - Updated README_SECURITY_GUARDRAILS_TESTGUIDE.md to cover Phases 1-5 Tests: ~107 new tests across 7 test files (252 total security tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace realistic-looking fake tokens in test files and docs with obviously-fake placeholders that won't trigger GitHub/GitGuardian secret scanning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MrBlack1995
force-pushed
the
feature/flow
branch
2 times, most recently
from
799ce21 to
b7480a9
Compare
When deploying to a non-DEFAULT Databricks CLI profile, the import-dir commands were missing the --profile flag, causing uploads to go to the wrong workspace. Cherry-picked from PR databrickslabs#32 by Felipe Cruz Neiva Campos, adapted to current hybrid upload strategy. Co-authored-by: FNeiva <2091157+FNeiva@users.noreply.github.com> Co-authored-by: Isaac
- New WHY_KASAL.md explaining what problems Kasal solves - Improved CODE_STRUCTURE_GUIDE.md with repo navigation - Updated API_REFERENCE.md with section descriptions - Added Kasal overview and logo images - Updated Documentation.tsx to include Why Kasal section Cherry-picked from PR databrickslabs#35 by Anshu Roy, adapted for feature/flow. Co-authored-by: Isaac
Prasad's PR databrickslabs#42 introduced the telemetry.py module and User-Agent attribution pattern. This was expanded by Simone (PR databrickslabs#43) and further extended across all Databricks API callers in PR databrickslabs#51. Co-authored-by: Isaac
Docker containerization idea from PR databrickslabs#44. While we opted not to include the Docker setup (local dev is already a 2-command process), we appreciate the contribution and the thinking behind lowering the onboarding barrier. Co-authored-by: Isaac
This was referenced Jun 3, 2026
LLMs occasionally return confidence > 1.0, crashing Pydantic validation in DispatcherResponse. Clamp to [0.0, 1.0] before creating the response. Cherry-picked from PR databrickslabs#45 by Stuart Gano. Co-authored-by: Isaac
Adds a complete blueprint showing how to integrate Kasal workflows with Databricks Genie: Genie retrieval → multi-agent analysis → executive summary. Includes setup guide, workflow tutorial, importable JSON, and screenshots. Cherry-picked from PR databrickslabs#49 by Amir Sahraei (.DS_Store excluded). Co-authored-by: Isaac
Rename the mode switcher label from "Crew" to "Agent Builder" for better clarity to non-technical users. Co-authored-by: Isaac
- Flow → Flow Builder for consistency - Groups (people) icon → SmartToy (robot) icon for Agent Builder Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Ensures frontend static assets are always fresh when deploying. No more forgetting to run build.py first. Co-authored-by: Isaac
Absolute path /databricks-logo.png fails when app is served under a subpath (e.g. Databricks Apps). Use process.env.PUBLIC_URL to resolve the correct base path. Co-authored-by: Isaac
Matches the working pattern used by other images (kasal-icon-16.png, kasal-icon-24.png). process.env.PUBLIC_URL doesn't work in this build. Co-authored-by: Isaac
The deployed app serves static files via explicit FastAPI routes in entrypoint.py. databricks-logo.png was missing its route, causing a 404 in chat mode. Added it alongside the existing kasal-icon routes. Co-authored-by: Isaac
nehmetohmedb
added a commit
that referenced
this pull request
Jun 4, 2026
…s + backend tests Integration fixes after merging PR #51 into app-modes, plus test coverage. Bug fixes: - Workspace member add failed: PR removed `hashed_password` from the User model, but two constructors still passed it (admin_auth._create_user_from_forwarded_email, group_service.assign_user_to_group) — every new-user create threw. Removed the stale kwarg. - `dev@localhost` (and other no-TLD local-dev emails) were rejected by the EmailStr field on GroupUserCreateRequest. Made user_email validation environment-aware: strict RFC validation in production/Databricks Apps, lenient (synthetic emails allowed) in local dev. - Fixed MUI console warnings surfaced by the workspace refresh: <Chip>/<Box> inside a <p> (GroupManagement ListItemText -> component:'div') and Tooltip-wrapped disabled buttons (RightSidebar, ExecutionHistory -> <span> wrappers). Merge test-fixes (components changed by PR #51): - ModeSwitcher labels renamed (Crew->Agent Builder, Flow->Flow Builder). - ShowResult/HITLApprovalDialog now use react-router; stub useNavigate in their unit tests (and ExecutionHistory.responsive, which renders ShowResult). Coverage: - ChatWorkspace.tsx brought to 100% (branches/statements/functions/lines); removed a debug log and simplified a few unreachable defensive branches. Threshold raised to 100 in vitest.config. - New backend unit tests (100% on each): ui_config model/schema/repository/service/ router, ui_emission, and the group user_email validator. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates all pending work from
feature/flowintoapp-modesfor the next release batch.Integrated PRs
--profilefixNew features & fixes
build.pyruns automatically as part ofdeploy.py--profileflag passed to CLI import-dir commandsContributors
Felipe Neiva, Anshu Roy, Prasad Kona, achasol-db, Stuart Gano, Amir Sahraei, Simone Finelli
Test plan
This pull request was AI-assisted by Isaac.