Planner initial impl

[expani]

Description

Adds the coordinator-side query planner for the Analytics Plugin — takes a logical RelNode tree from the DSL converter and produces a marked, optimized plan with a
QueryDAG ready for distributed execution.

What this PR adds along with existing changes from https://github.com/expani/OpenSearch/tree/planner_initial_impl:

1. SPI Capability Model (analytics-framework)

• FilterCapability (sealed: Standard, FullText, Expression) scoped to operator + field type + format
• AggregateCapability, ProjectCapability (Scalar, Opaque), WindowCapability — all format-aware
• FieldType enum with granular types (INTEGER, KEYWORD, TEXT, etc.) and convenience groupings
• OperatorCapability extended with PROJECT, WINDOW, COORDINATOR_REDUCE
• DelegationType and ShuffleCapability for cross-backend delegation and shuffle support
• Temporary default for createSearchExecEngine() on AnalyticsSearchBackendPlugin (TODO: separate capability declaration from execution engine factory)

2. Planner Pipeline (analytics-engine)

• Two-phase planner: HepPlanner (RBO) for LogicalXxx → OpenSearchXxx marking, VolcanoPlanner (CBO) for trait propagation and exchange insertion
• CapabilityRegistry — singleton built at plugin startup, lazy-cached lookups for filter/aggregate/scalar/opaque/window backends by (operator, fieldType, format)
• FieldStorageResolver — resolves per-field storage (doc values, index, stored fields) from IndexMetadata + backend DataFormat capabilities
• Five marking rules: TableScan, Filter, Project, Aggregate, Sort — all use CapabilityRegistry, no backend iteration in rules
• Per-expression annotations: AnnotatedPredicate, AggregateCallAnnotation, AnnotatedProjectExpression — each carries viableBackends for downstream plan forking
• Operator-level viableBackends propagation constrained by delegation capability
• Aggregate split rule (PARTIAL + FINAL) via CBO when distribution trait mismatches (RANDOM → SINGLETON)

3. DAG Construction

• DAGBuilder cuts the CBO output at exchange boundaries (ExchangeReducer for SINGLETON, ShuffleReader/ExchangeWriter for HASH)
• Stage, QueryDAG, ExchangeInfo data structures — recursive tree of stages
• OpenSearchStageInputScan replaces severed subtrees with stage references
• Stage IDs assigned bottom-up, fragments retain annotations for downstream plan forking

4. Tests (34 planner tests across 3 suites)

• AggregateRuleTests (12): per-call annotation, viableBackends intersection, delegation, split on multi-shard, no-split on single-shard, DAG verification
• FilterRuleTests (9): native predicates, full-text delegation, AND/OR across backends, derived column filters, error cases
• ProjectRuleTests (13): field projection, scalar functions, opaque ops (painless, highlight), delegation, nested expressions, error cases
• MockDataFusionBackend / MockLuceneBackend — reusable test backends with realistic capabilities

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

What's Next

Based on the implementation plan, here's the roadmap after this PR:

Immediate (before next PR):

1. Hybrid index tests — both backends with COLUMNAR_STORAGE for same field, verify viableBackends=[DF, Lucene] propagates end-to-end
2. Plan forking / multiple plan generation — per-stage, generate one StagePlan per viable backend, walk fragment to resolve annotations into clean backend-specific plans,
   attach CostFunctionDescriptor
3. StagePlan + DelegatedBackendPlan + CostFunctionDescriptor data structures
4. Built-in cost functions — AlwaysViable, SelectivityBased, PrecomputeViable

Phase 3: PlanWalker + Scheduler (coordinator execution)

• Reactive DAG walker that dispatches tasks bottom-up
• Scheduler with concurrency control and task queuing
• Transport integration (AnalyticsTaskAction for sending FragmentExecutionRequest to data nodes)
• Thread pool registration (analytics_outbound, analytics_reduce)

Phase 4: Data node execution

• TaskReceiver — receives FragmentExecutionRequest, registers sources, calls backend
• Backend SPI for data node (registerSource, convertAndOptimize, execute)
• DataFusion backend implementation (Substrait → LogicalPlan → PhysicalPlan)

Phase 5: E2E wiring

• Wire PlannerImpl → DAG → Scheduler → PlanWalker → TaskDispatch → DataNode → Results
• Integration test: SELECT status, SUM(size) FROM http_logs GROUP BY status HAVING SUM(size) > 1000000

Known TODOs / tech debt:

• Separate AnalyticsSearchBackendPlugin (capability declaration) from SearchExecEngineProvider (execution factory)
• FieldStorageResolver infers from capabilities, not actual per-field format metadata (blocked on indexing team)
• Scan viable backends uses union instead of per-field coverage check
• Stored fields integration for late materialization / fetch phase