From 7e4adf5cc45e5d0aa5ca57f7d43283d68a428d71 Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 13:52:44 +0000 Subject: [PATCH 1/6] Value Operations Discovery Audit Performed a complete discovery audit of value manipulation capabilities in FlexiRule. Traced operations from FlexValueControl and registries to their backend implementations. Documented capability inventory, runtime mappings, duplicates, and functional overlaps. From 07e8610daa603755e43cc6ac47ef869f69efd086 Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 13:56:14 +0000 Subject: [PATCH 2/6] Value Operations Discovery Audit Report Added `VALUE_OPERATIONS_AUDIT.md` containing the complete discovery audit results. Traced all frontend operations to backend implementations. Documented inventory, mapping, duplicates, and functional overlaps. --- VALUE_OPERATIONS_AUDIT.md | 129 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 VALUE_OPERATIONS_AUDIT.md diff --git a/VALUE_OPERATIONS_AUDIT.md b/VALUE_OPERATIONS_AUDIT.md new file mode 100644 index 00000000..5a2ecbbb --- /dev/null +++ b/VALUE_OPERATIONS_AUDIT.md @@ -0,0 +1,129 @@ +# Value Operations Discovery Audit Report + +## A. Capability Inventory + +This inventory lists all value manipulation capabilities exposed in FlexiRule, categorized by their entry point. + +### 1. Structured Value Resolvers (via `ValueResolverControl`) +| Kind | Name | Description | Sub-Operations | +| :--- | :--- | :--- | :--- | +| `date_formula` | Date Formula | Calculate a date by adding or subtracting days, months, or years from a base field or today. | Add/Subtract Days, Months, Years | +| `date_diff` | Date Difference | Calculate the time difference between two dates in days, months, or years. | Days, Months, Years | +| `math_formula` | Math Formula | Perform basic arithmetic between two fields or a field and a constant value. | `+`, `-`, `*`, `/` | +| `child_aggregation` | Child Table Aggregation | Aggregate numeric values from a child table using Sum, Average, or Count. | Sum, Average, Count | +| `string_formula` | String Manipulation | Combine text fields, change casing, or format currency strings. | Concatenate, Uppercase, Lowercase, Format Currency | +| `normalization` | Normalization | Clean up text data by trimming whitespace, changing case, or converting to slug/snake case. | Pipeline of 25+ operations (Trim, Slug, Masking, etc.) | +| `format` | Format | Presentation-level formatting for dates, currency, and templates. | Date/Time, Currency, String Template | +| `fetch` | Fetch From Link | Fetch a value from a linked document. | Database lookup across DocTypes | +| `system_context` | System Context | Resolve global values like current user or role status. | Current User, Role Check | + +### 2. Formula Registry & Slash Commands (`formula_registry.js`) +These are exposed as autocomplete suggestions or slash commands in the `FlexValueControl`. + +**Slash Commands (Value-Related):** +- `/formula`: Opens logic selection. +- `/resolver`: Opens structured resolver. +- `/formatter`: Opens formatting UI. +- `/normalize`: Opens normalization UI. +- `/fetch`: Opens link fetching UI. +- `/link`: Opens link picker. +- `/boolean`: Opens toggle UI. + +**Formula Registry Items (by Category):** +- **TEXT:** `concat`, `normalize`, `format`, `replace`, `trim`, `upper`, `lower`, `slug`. +- **NUMERIC:** `sum`, `round`, `avg`, `min`, `max`, `percentage`, `abs`. +- **DATE:** `today`, `now`, `add_days`, `add_months`, `date_diff`, `format_date`, `start_of`, `end_of`. +- **BOOLEAN:** `if`, `equals`, `not`. +- **LINK:** `lookup`. +- **TABLE:** `sum`, `count`, `map`, `filter`, `reduce`. + +### 3. Normalization Operations (`normalization.py`) +Exposed primarily through the `NormalizationResolver` pipeline. + +- **Cleaning:** `trim`, `remove_spaces`, `remove_extra_spaces`, `remove_punctuation`, `remove_numbers`. +- **Casing:** `lowercase`, `uppercase`, `casefold`, `title_case`, `name_normalize`. +- **Transformations:** `slug`, `snake_case`, `unicode_normalize`, `remove_diacritics`, `translate_chars` (Arabic/Persian digits & characters). +- **Specialized:** `phone_normalize`, `email_normalize`, `currency_to_number`, `tax_id_clean`, `standard_date`. +- **Masking:** `mask_email`, `mask_phone`, `mask_credit_card`, `mask_partial`. + +--- + +## B. Runtime Mapping + +This table traces frontend capabilities to their backend execution logic. + +| Frontend Identifier | Backend Implementation Path | Actual Execution Logic | +| :--- | :--- | :--- | +| `date_formula` | `ValueResolver.compile` → `DateFormulaResolver.resolve` | `frappe.utils.add_days` or `frappe.utils.add_to_date` | +| `math_formula` | `ValueResolver.compile` → `MathFormulaResolver.resolve` | `frappe.utils.flt(a op b, precision)` | +| `date_diff` | `ValueResolver.compile` → `DateDiffResolver.resolve` | `frappe.utils.date_diff` or `frappe.utils.month_diff` | +| `child_aggregation`| `ValueResolver.compile` → `ChildAggregationResolver.resolve` | List comprehension over `doc.get(table)` with `sum()`/`len()` | +| `string_formula` | `ValueResolver.compile` → `StringFormulaResolver.resolve` | Python `str.upper()`, `str.lower()`, or `frappe.utils.fmt_money` | +| `normalization` | `ValueResolver.compile` → `NormalizationResolver.resolve` | `flexirule.ruleflow.utils.normalization.execute_normalization_pipeline` | +| `format` | `ValueResolver.compile` → `FormatResolver.resolve` | `frappe.utils.format_date`, `frappe.utils.fmt_money`, or `"".format()` | +| `fetch` | `ValueResolver.compile` → `FetchResolver.resolve` | `frappe.db.get_value` | +| `system_context` | `ValueResolver.compile` → `SystemContextResolver.resolve` | `frappe.session.user` or `frappe.get_roles` | +| `today()` (Formula) | `RuleEngine._build_eval_locals` | `frappe.utils.nowdate` mapped as `nowdate` in context | +| `add_days()` (Formula)| `RuleEngine._build_eval_locals` | `frappe.utils.add_days` mapped as `add_days` in context | +| `sum()` (Formula) | `ActionHandler._build_template_context` | Python built-in `sum()` | +| `normalize()` (Jinja)| `ActionHandler._build_template_context` | `flexirule.ruleflow.process.normalization.normalization.apply_transformations` | + +--- + +## C. Duplicate List (Exact Duplicates) + +Operations that use identical underlying logic but are exposed under different names or categories: + +1. **Currency Formatting:** + - `String Formula` resolver → `fmt_money` + - `Format` resolver → `fmt_money` + - *Both call `frappe.utils.fmt_money`.* +2. **Basic Casing:** + - `String Formula` resolver → `uppercase`/`lowercase` + - `Normalization` resolver → `uppercase`/`lowercase` pipeline steps + - *Both call Python `.upper()` / `.lower()`.* +3. **Date Difference:** + - `Date Difference` resolver + - `/formula` suggestion for `date_diff()` + - *Both call `frappe.utils.date_diff`.* + +--- + +## D. Similarity List (Functional Overlaps) + +Operations that produce substantially similar results through different mechanisms: + +1. **Concatenation:** + - `String Formula` resolver: 2-field visual builder. + - `concat()` Formula: Variadic text function. + - `String Template` (Format resolver): Uses `"{field_a}{field_b}".format()`. +2. **Date Math:** + - `Date Formula` resolver: Step-based UI (Add 5 days). + - `add_days()` / `add_months()` Formulas: Functional approach. +3. **Normalization vs Formulas:** + - `trim()`, `upper()`, `lower()`, `slug()` exist both as standalone formulas and as steps in the `Normalization` pipeline. +4. **Aggregation:** + - `Child Table Aggregation` resolver: Visual UI for Sum/Avg/Count. + - `sum()`, `count()`, `len()` formulas: Direct access to Python aggregations. + +--- + +## E. Observations + +### 1. Registry vs. Runtime Gaps +The `formula_registry.js` contains many items (like `map`, `reduce`, `slug`) that are primarily UI "hints." There is no single centralized "Formula Library" on the backend; instead, the `RuleEngine` and `ActionHandler` manually inject various helpers into different evaluation contexts (Python Eval vs. Jinja), leading to inconsistent availability of functions depending on where they are used. + +### 2. Architectural Redundancy +The `String Formula` resolver is largely a subset of the `Format` and `Normalization` resolvers. It appears to be a legacy entry point that has been superseded by more specialized strategies. + +### 3. Hidden Backend Power +The `normalization.py` utility contains sophisticated logic (Arabic character unification, diacritic removal, various masking strategies) that is only fully accessible through the `Normalization` resolver's "Custom Pipeline" mode. These capabilities are not exposed in the simpler `/formula` or `/formatter` registries. + +### 4. Implementation Leakage +Several frontend strategies (e.g., `Aggregation`) "leak" implementation details by compiling complex Python list comprehensions in the frontend `compileToCode` function. This makes it difficult to change the aggregation logic (e.g., adding null checks) without updating the frontend registry. + +### 5. Inconsistent Scoping +The `Fetch` resolver defaults to `doc.` scoping if no scope is provided, while the `Formula` system requires explicit scoping or relies on the evaluator's local resolution logic. + +### 6. Special Case Context +The `Assignment` action handler injects a special `value` variable into the evaluation context specifically for Normalization and Formatting steps. This "current value" awareness is not consistently available across other action types or resolvers. From c0d88b33f610c2f2a082c1f6b0d8cd860176810a Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 14:00:51 +0000 Subject: [PATCH 3/6] Value Operations Discovery Audit - Capability Coverage Matrix Updated `VALUE_OPERATIONS_AUDIT.md` with: - Comprehensive Capability Coverage Matrix. - Detailed analysis of multi-source vs single-source capabilities. - Identification of UX wrappers and hidden specialized operations. - Tracing of all unique operations to backend implementations. --- VALUE_OPERATIONS_AUDIT.md | 64 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/VALUE_OPERATIONS_AUDIT.md b/VALUE_OPERATIONS_AUDIT.md index 5a2ecbbb..22078c04 100644 --- a/VALUE_OPERATIONS_AUDIT.md +++ b/VALUE_OPERATIONS_AUDIT.md @@ -127,3 +127,67 @@ The `Fetch` resolver defaults to `doc.` scoping if no scope is provided, while t ### 6. Special Case Context The `Assignment` action handler injects a special `value` variable into the evaluation context specifically for Normalization and Formatting steps. This "current value" awareness is not consistently available across other action types or resolvers. + +--- + +## F. Capability Coverage Matrix + +Mapping of unique value operations to their availability across different entry points. + +| Capability | Formula | String | Normalize | Format | Date | Math | Aggreg | Fetch | System | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| **TEXT** | | | | | | | | | | +| trim | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| lowercase | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| uppercase | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| title_case | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| concat | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| slug | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| snake_case | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| template format | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | +| remove_spaces | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| remove_punctuation | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| **NUMERIC** | | | | | | | | | | +| sum | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | +| average | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | +| count | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | +| arithmetic (+,-,*,/) | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | +| round | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | +| currency format | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | +| **DATE/TIME** | | | | | | | | | | +| today / now | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | +| add_days/months | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | +| date_diff | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| format_date | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | +| **SPECIALIZED** | | | | | | | | | | +| phone_normalize | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| email_normalize | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| mask_email/phone | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| Arabic unification | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| lookup / fetch | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | +| current_user | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | +| role_check | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | + +--- + +## G. Matrix Analysis + +### 1. Multi-Source Capabilities +Capabilities like **lowercase**, **uppercase**, **sum**, and **average** are heavily distributed, appearing in 3 or more places. This suggests high demand but inconsistent implementation across the UI. + +### 2. Single-Source Capabilities +- **Masking** and **Arabic unification** are exclusive to the Normalization Resolver. +- **Arithmetic operators** are exclusive to the Math Resolver (when using the builder) or raw Formulas. +- **Role checks** are exclusive to the System Resolver. +- **Fetch** logic is centralized in the Fetch Resolver. + +### 3. Resolvers with No Unique Capability +The **String Formula Resolver** contributes no unique capabilities. Everything it does (concat, upper, lower, fmt_money) is available through either the Normalization Resolver, Format Resolver, or direct Formulas. + +### 4. UX Wrappers +- The **Math Resolver** is essentially a UX wrapper for Python's `flt()` and arithmetic operators. +- The **Date Resolver** is a UX wrapper for `frappe.utils.add_days` and `add_months`. +- The **Child Aggregation Resolver** is a UX wrapper for list comprehensions. + +### 5. Hidden Capabilities +The **Normalization Resolver** contains many specialized operations (Phone/Email normalization, Punctuation removal, Masking, Arabic unification) that are powerful but hidden from the general "Format" or "Formula" user journeys. These are currently "locked" behind a specific resolver kind, making them inaccessible for users who don't think to look under "Normalization." From 9a2fa29ca41716eccd3a75f36504ab17b26d44ae Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 14:13:52 +0000 Subject: [PATCH 4/6] Value Operations: Discovery Audit and Architecture Refactor Proposal This PR delivers a complete discovery audit of all value manipulation capabilities in FlexiRule, followed by a proposed architectural refactor to unify these operations. 1. Added `VALUE_OPERATIONS_AUDIT.md`: - Complete inventory of Resolvers, Formulas, and Normalization ops. - Trace mapping from Frontend entry points to Backend runtime implementation. - Capability Coverage Matrix identifying overlaps and gaps. 2. Added `VALUE_OPERATIONS_REFACTOR.md`: - Evaluation matrix for existing resolvers (Keep/Merge/Remove). - Proposed intent-based Capability Taxonomy. - High-level design for a unified Registry-bound Backend and Frontend architecture. - Vision for a "Command Palette First" discovery UX. No functional code changes were made as per the task requirements. --- VALUE_OPERATIONS_REFACTOR.md | 88 ++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 VALUE_OPERATIONS_REFACTOR.md diff --git a/VALUE_OPERATIONS_REFACTOR.md b/VALUE_OPERATIONS_REFACTOR.md new file mode 100644 index 00000000..d425b727 --- /dev/null +++ b/VALUE_OPERATIONS_REFACTOR.md @@ -0,0 +1,88 @@ +# FlexiRule Value Operations Architecture Refactor (v2) + +This document outlines the proposed architectural refactor for Value Operations in FlexiRule, moving from a fragmented "Resolver-Kind" system to a unified, capability-driven "Operation Registry." + +--- + +## 1. Resolver Evaluation Matrix + +Challenge the status quo of existing resolver boundaries. + +| Resolver Type | Purpose | Unique Capabilities | Overlap | Recommendation | Justification | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **String Formula** | Simple text/currency ops | None | Normalization, Format, Formulas | **Remove / Merge** | Redundant. Its logic exists in more specialized or functional entry points. | +| **Math Formula** | Simple arithmetic | None | Formulas, Python `flt()` | **Remove / Merge** | Pure UX wrapper for basic Python operators. Should be atomic operations in a registry. | +| **Date Formula** | Date arithmetic | None | Formulas, `frappe.utils` | **Remove / Merge** | UX wrapper. "Add Days" is an operation, not a standalone architecture category. | +| **Normalization** | Data cleaning/standardization | Arabic unification, specialized cleaning | Casing, slugging | **Keep & Rename** | Purpose is valid, but should be renamed to **"Transformation"** and expanded. | +| **Format** | Presentation logic | String Templates, Locale-aware formatting | Currency formatting | **Keep** | Solves the specific problem of *display* vs *value*, which is conceptually distinct. | +| **Child Aggregation** | List math | Table-to-Scalar reduction | `sum`, `count` formulas | **Keep** | Essential for handling list context which requires specific configuration UI. | +| **Fetch** | Remote data | Cross-DocType retrieval | Lookup formula | **Keep** | Distinct capability requiring unique UI for DocType/Field selection. | +| **System Context** | Global state | Role checking, session data | None | **Keep** | Unique source of truth (the environment, not the document). | + +--- + +## 2. Proposed Capability Taxonomy + +Operations should be grouped by **author intent**, not implementation logic. + +1. **Text Transformation:** Trimming, casing, slugging, snake_case, find/replace. +2. **Numeric & Math:** Arithmetic, rounding, absolute value, percentages. +3. **Date & Time:** Adding/subtracting time units, date differences, relative dates (today, now). +4. **Data Normalization:** Cleaning phone numbers, emails, tax IDs, Arabic/Persian unification. +5. **Masking & Privacy:** Masking emails, credit cards, or partial strings (security-centric). +6. **Formatting & Display:** Currency formatting, date/time formatting, string templates. +7. **Collection & Aggregation:** Sum, average, count, min/max across child tables. +8. **Data Retrieval:** Fetching values from linked documents or remote DocTypes. +9. **Environment & Context:** Current user, role checks, rule metadata. + +--- + +## 3. Proposed Backend Architecture + +**Core Principle:** Atomic, registry-bound operations. + +- **`flexirule.ruleflow.operations.registry`**: A centralized singleton registry that maps `operation_id` to an implementation class. +- **`ValueOperation` (Base Class)**: + - `execute(value, context, config)`: Standard interface. + - `get_meta()`: Returns required parameters, return type, and description. +- **Module Structure:** + - `operations/text.py` + - `operations/math.py` + - `operations/date.py` + - `operations/normalization.py` (Centralizes all logic from `normalization.py` and `NormalizationResolver`). +- **Compiler**: `ValueResolver.compile` no longer needs kind-specific logic. It simply looks up the `operation_id` in the registry and builds a standard execution chain. + +--- + +## 4. Proposed Frontend Architecture + +**Core Principle:** Discoverability through unified browsing. + +- **`ValueRegistry`**: A shared registry between `formula_registry.js` and `ValueResolverControl`. +- **`OperationBrowser.vue`**: A "Command Palette" style component used in `FlexValueControl` that allows searching all categories at once. +- **`GenericOperationConfigurator.vue`**: A metadata-driven UI component that renders inputs based on the operation's parameter definitions (Schema-driven UI). + - *Example:* If "Add Days" is selected, it renders a "Base Date" picker and a "Number" input. +- **Specific Configurators**: Only reserved for complex logic like `Aggregation` (Table selection) or `Fetch` (Link selection). + +--- + +## 5. Recommended End-State (v2) + +In FlexiRule v2, the concept of selecting a "Resolver Type" first is eliminated. + +### The Author Experience: +1. User clicks into a value field. +2. User types `/` to open the **Operation Palette**. +3. User searches for what they want to *do* (e.g., "mask", "sum", "days"). +4. FlexiRule displays a unified list of matches: + - **Add Days** (Date & Time) + - **Mask Email** (Masking & Privacy) + - **Sum Child Table** (Aggregation) +5. Selection opens a consistent configuration flyout. +6. The token in the editor is simply an **Operation Token** (e.g., `[Add 5 Days]`). + +### Why this is better: +- **Zero dead ends:** Users don't need to know if "Slug" is a "Normalization" or a "String Formula." +- **Higher discoverability:** Powerful features like "Arabic Unification" or "Partial Masking" are surfaced alongside basic tasks. +- **Maintainability:** Adding a new capability involves adding one class on the backend and one entry in the registry. The UI handles the rest. +- **Consistency:** All value manipulation follows the same "Search -> Configure -> Tokenize" lifecycle. From c73dfee5529390386fbcbfc1b9960811a61dbc8d Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 14:52:43 +0000 Subject: [PATCH 5/6] Value Operations: Discovery Audit and v2 Architecture Strategy Added `VALUE_OPERATIONS_AUDIT.md`: - Complete inventory and trace mapping of 55+ operations. - Capability Coverage Matrix identifying significant architectural overlaps. Added `VALUE_OPERATIONS_REFACTOR.md`: - Strategy for merging redundant resolvers (String, Math, Date) into a unified registry. - Proposed intent-based taxonomy (Text Ops, Normalization, Masking, Formatting). - Unified non-unary backend API: `execute(context, config)`. - Metadata-driven frontend configurator for improved discoverability. --- VALUE_OPERATIONS_REFACTOR.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/VALUE_OPERATIONS_REFACTOR.md b/VALUE_OPERATIONS_REFACTOR.md index d425b727..a3ec207f 100644 --- a/VALUE_OPERATIONS_REFACTOR.md +++ b/VALUE_OPERATIONS_REFACTOR.md @@ -13,7 +13,7 @@ Challenge the status quo of existing resolver boundaries. | **String Formula** | Simple text/currency ops | None | Normalization, Format, Formulas | **Remove / Merge** | Redundant. Its logic exists in more specialized or functional entry points. | | **Math Formula** | Simple arithmetic | None | Formulas, Python `flt()` | **Remove / Merge** | Pure UX wrapper for basic Python operators. Should be atomic operations in a registry. | | **Date Formula** | Date arithmetic | None | Formulas, `frappe.utils` | **Remove / Merge** | UX wrapper. "Add Days" is an operation, not a standalone architecture category. | -| **Normalization** | Data cleaning/standardization | Arabic unification, specialized cleaning | Casing, slugging | **Keep & Rename** | Purpose is valid, but should be renamed to **"Transformation"** and expanded. | +| **Normalization** | Data cleaning/standardization | Arabic unification, specialized cleaning | Casing, slugging | **Keep** | Highly specialized domain with deep logic that shouldn't be diluted into general transformation. | | **Format** | Presentation logic | String Templates, Locale-aware formatting | Currency formatting | **Keep** | Solves the specific problem of *display* vs *value*, which is conceptually distinct. | | **Child Aggregation** | List math | Table-to-Scalar reduction | `sum`, `count` formulas | **Keep** | Essential for handling list context which requires specific configuration UI. | | **Fetch** | Remote data | Cross-DocType retrieval | Lookup formula | **Keep** | Distinct capability requiring unique UI for DocType/Field selection. | @@ -23,14 +23,14 @@ Challenge the status quo of existing resolver boundaries. ## 2. Proposed Capability Taxonomy -Operations should be grouped by **author intent**, not implementation logic. +Operations should be grouped by **author intent**, maintaining distinct domains for specialized data handling. -1. **Text Transformation:** Trimming, casing, slugging, snake_case, find/replace. +1. **Text Operations:** Trimming, casing, slugging, snake_case, find/replace, concatenation. 2. **Numeric & Math:** Arithmetic, rounding, absolute value, percentages. 3. **Date & Time:** Adding/subtracting time units, date differences, relative dates (today, now). 4. **Data Normalization:** Cleaning phone numbers, emails, tax IDs, Arabic/Persian unification. 5. **Masking & Privacy:** Masking emails, credit cards, or partial strings (security-centric). -6. **Formatting & Display:** Currency formatting, date/time formatting, string templates. +6. **Formatting & Localization:** Currency formatting, date/time formatting, string templates, locale overrides. 7. **Collection & Aggregation:** Sum, average, count, min/max across child tables. 8. **Data Retrieval:** Fetching values from linked documents or remote DocTypes. 9. **Environment & Context:** Current user, role checks, rule metadata. @@ -39,18 +39,19 @@ Operations should be grouped by **author intent**, not implementation logic. ## 3. Proposed Backend Architecture -**Core Principle:** Atomic, registry-bound operations. +**Core Principle:** Registry-bound operations with flexible input signatures. - **`flexirule.ruleflow.operations.registry`**: A centralized singleton registry that maps `operation_id` to an implementation class. - **`ValueOperation` (Base Class)**: - - `execute(value, context, config)`: Standard interface. + - `execute(context, config)`: Standard non-unary interface. Configuration handles all specific inputs (including target values if applicable). - `get_meta()`: Returns required parameters, return type, and description. - **Module Structure:** - `operations/text.py` - `operations/math.py` - `operations/date.py` - - `operations/normalization.py` (Centralizes all logic from `normalization.py` and `NormalizationResolver`). -- **Compiler**: `ValueResolver.compile` no longer needs kind-specific logic. It simply looks up the `operation_id` in the registry and builds a standard execution chain. + - `operations/normalization.py` + - `operations/masking.py` +- **Compiler**: `ValueResolver.compile` simply looks up the `operation_id` in the registry and builds a standard execution call passing the context and pre-parsed config. --- @@ -74,15 +75,15 @@ In FlexiRule v2, the concept of selecting a "Resolver Type" first is eliminated. 1. User clicks into a value field. 2. User types `/` to open the **Operation Palette**. 3. User searches for what they want to *do* (e.g., "mask", "sum", "days"). -4. FlexiRule displays a unified list of matches: +4. FlexiRule displays a unified list of matches categorized by domain: - **Add Days** (Date & Time) - **Mask Email** (Masking & Privacy) - - **Sum Child Table** (Aggregation) + - **Normalize Phone** (Data Normalization) 5. Selection opens a consistent configuration flyout. 6. The token in the editor is simply an **Operation Token** (e.g., `[Add 5 Days]`). ### Why this is better: -- **Zero dead ends:** Users don't need to know if "Slug" is a "Normalization" or a "String Formula." -- **Higher discoverability:** Powerful features like "Arabic Unification" or "Partial Masking" are surfaced alongside basic tasks. +- **Conceptual Clarity:** Domains like "Normalization" and "Masking" are preserved as distinct concepts for the user. +- **Flexible API:** Supporting `execute(context, config)` allows for N-ary operations (Concat), null-input operations (System Context), and complex retrieval (Fetch) without hacking a unary `value` parameter. - **Maintainability:** Adding a new capability involves adding one class on the backend and one entry in the registry. The UI handles the rest. - **Consistency:** All value manipulation follows the same "Search -> Configure -> Tokenize" lifecycle. From dc37e3aeaf77995240fe369dd00beb6c2426c389 Mon Sep 17 00:00:00 2001 From: ruzaqiarkan-eng <289478799+ruzaqiarkan-eng@users.noreply.github.com> Date: Thu, 18 Jun 2026 14:56:39 +0000 Subject: [PATCH 6/6] Value Operations: Discovery Audit and Target Architecture Strategy Added `VALUE_OPERATIONS_AUDIT.md`: - Complete discovery inventory and trace mapping of 55+ operations. - Capability Coverage Matrix highlighting overlaps in current resolvers. Added `VALUE_OPERATIONS_REFACTOR.md`: - Strategic plan for merging redundant resolvers into a unified registry. - Intent-based taxonomy: Text Ops, Data Normalization, Masking, Formatting. - Target Backend API: `execute(context, config)` for non-unary support. - Hybrid Frontend UX: Metadata-driven forms for simple ops, dedicated Vue components for complex ones (Fetch, Aggregation, etc.). --- VALUE_OPERATIONS_REFACTOR.md | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/VALUE_OPERATIONS_REFACTOR.md b/VALUE_OPERATIONS_REFACTOR.md index a3ec207f..852f1d4e 100644 --- a/VALUE_OPERATIONS_REFACTOR.md +++ b/VALUE_OPERATIONS_REFACTOR.md @@ -1,6 +1,6 @@ -# FlexiRule Value Operations Architecture Refactor (v2) +# FlexiRule Value Operations Architecture Refactor -This document outlines the proposed architectural refactor for Value Operations in FlexiRule, moving from a fragmented "Resolver-Kind" system to a unified, capability-driven "Operation Registry." +This document outlines the proposed target architecture for Value Operations in FlexiRule, moving from a fragmented "Resolver-Kind" system to a unified, capability-driven "Operation Registry." --- @@ -57,19 +57,20 @@ Operations should be grouped by **author intent**, maintaining distinct domains ## 4. Proposed Frontend Architecture -**Core Principle:** Discoverability through unified browsing. +**Core Principle:** Hybrid configuration for optimal UX. - **`ValueRegistry`**: A shared registry between `formula_registry.js` and `ValueResolverControl`. - **`OperationBrowser.vue`**: A "Command Palette" style component used in `FlexValueControl` that allows searching all categories at once. -- **`GenericOperationConfigurator.vue`**: A metadata-driven UI component that renders inputs based on the operation's parameter definitions (Schema-driven UI). - - *Example:* If "Add Days" is selected, it renders a "Base Date" picker and a "Number" input. -- **Specific Configurators**: Only reserved for complex logic like `Aggregation` (Table selection) or `Fetch` (Link selection). +- **Hybrid Configuration System**: + - **Metadata-Driven Renderer**: For simple operations (e.g., Round, Trim, Add Days), the UI is automatically generated from the operation's metadata/schema. + - **Dedicated Vue Components**: For complex operations where a generic form is insufficient (e.g., `Aggregation`, `Fetch`, `System Context`, `Template Formatting`), the registry allows mapping an `operation_id` to a specialized Vue configuration component. +- **Orchestrator**: A unified modal that determines whether to show the generic renderer or the dedicated component based on the selected operation. --- -## 5. Recommended End-State (v2) +## 5. Recommended End-State -In FlexiRule v2, the concept of selecting a "Resolver Type" first is eliminated. +The new architecture eliminates the need for users to understand "Resolver Kinds" before they start searching for functionality. ### The Author Experience: 1. User clicks into a value field. @@ -79,11 +80,13 @@ In FlexiRule v2, the concept of selecting a "Resolver Type" first is eliminated. - **Add Days** (Date & Time) - **Mask Email** (Masking & Privacy) - **Normalize Phone** (Data Normalization) -5. Selection opens a consistent configuration flyout. -6. The token in the editor is simply an **Operation Token** (e.g., `[Add 5 Days]`). +5. Selection opens the configuration flyout: + - *Simple ops:* Use the generic renderer. + - *Complex ops:* Use the dedicated Vue component. +6. The token in the editor is an **Operation Token** (e.g., `[Add 5 Days]`). ### Why this is better: -- **Conceptual Clarity:** Domains like "Normalization" and "Masking" are preserved as distinct concepts for the user. -- **Flexible API:** Supporting `execute(context, config)` allows for N-ary operations (Concat), null-input operations (System Context), and complex retrieval (Fetch) without hacking a unary `value` parameter. -- **Maintainability:** Adding a new capability involves adding one class on the backend and one entry in the registry. The UI handles the rest. -- **Consistency:** All value manipulation follows the same "Search -> Configure -> Tokenize" lifecycle. +- **No "Type-First" Friction:** Users don't need to know if "Slug" is a "Normalization" or a "String Formula." +- **Flexible API:** Supporting `execute(context, config)` allows for N-ary operations (Concat), null-input operations (System Context), and complex retrieval (Fetch). +- **Maintainability:** Most new operations can be added via metadata alone. Dedicated components are only written when the UX requires a specialized interface. +- **Discoverability:** Surfaces powerful hidden capabilities (Arabic Unification, Masking) alongside standard ones.