v0.5.0: aggregates, BEAM callbacks, magic sets, query planner, runtim…

[thanos]

…e facts

M0.5 — Feasibility Spikes

• Spike A: callback as {:callback, %IR.Callback{}} IR literal round-trips through compiler/evaluator (chest/spikes/README.md)
• Spike B: materialize(goal:) drives magic sets transform; results ≡ semi-naive

M1 — Query Planner Foundation - closed #127

• Add ExDatalog.Planner with plan/2, explain_plan/1,2 - closed Add ExDatalog.Planner with plan/2, explain_plan/1,2 #128
• Add planner structs: Plan, Stratum, Join, Predicate (lib/ex_datalog/planner/) - closed Add planner structs: Plan, Stratum, Join, Predicate (lib/ex_datalog/planner/) #129
• Engine strategy dispatch: evaluate/2 routes :semi_naive / :magic_sets - closed Engine strategy dispatch: evaluate/2 routes :semi_naive / :magic_sets #130
• Telemetry [:ex_datalog, :planner, :start|:stop|:exception] - closed Telemetry [:ex_datalog, :planner, :start|:stop|:exception] #131
• Tests: planner_test.exs, planner_explain_test.exs - closed Tests: planner_test.exs, planner_explain_test.exs #132

M2 — Aggregates (count/sum/min/max) - closed #133

• Add @aggregate_ops [:count, :sum, :min, :max] to Constraint;
  from_tuple/1, valid_right?/2, valid_result?/2 extended - closed Add @aggregate_ops [:count, :sum, :min, :max] to Constraint; from_tuple/1, valid_right?/2, valid_result?/2 extended #134
• Add ExDatalog.Constraints.Aggregate (group_and_reduce/5; evaluate/3 raises) - closed Add ExDatalog.Constraints.Aggregate (group_and_reduce/5; evaluate/3 raises) #135
• Add Rule.has_aggregates?/1, aggregate_constraints/1, non_aggregate_constraints/1 - closed Add Rule.has_aggregates?/1, aggregate_constraints/1, non_aggregate_constraints/1 #136
• Safety: aggregate input var must be bound, result var in head, ≤1 agg/rule,
  no aggregate in recursive rule - closed Safety: aggregate input var must be bound, result var in head, ≤1 agg/rule, no aggregate in recursive rule #137
• Stratification: post-pass in assign_strata/1 forces aggregate head above
  all positive body relations; engine validate_stratification/1 extended - closed Stratification: post-pass in assign_strata/1 forces aggregate head above all positive body relations; engine validate_stratification/1 extended #138
• Evaluator: eval_aggregate_rule/3 (join → filter → callbacks → negation →
  group_and_reduce → project); apply_constraints/3 partitions out aggregate ops - closed Evaluator: eval_aggregate_rule/3 (join → filter → callbacks → negation →group_and_reduce → project); apply_constraints/3 partitions out aggregate ops #139
• DSL: count/sum/min/max(X, Result) parsed before generic constraint-ops loop;
  head-position aggregate raises DSL.CompileError - closed DSL: count/sum/min/max(X, Result) parsed before generic constraint-ops loop;head-position aggregate raises DSL.CompileError #140
• Tests: aggregate_test.exs + DSL tests in schema_test.exs - closed Tests: aggregate_test.exs + DSL tests in schema_test.exs #141

M3 — BEAM Callback Predicates - closed #142

• Add ExDatalog.Callback struct (AST) + IR.Callback struct + {:callback, _} IR literal - closed Add ExDatalog.Callback struct (AST) + IR.Callback struct + {:callback, _} IR literal #143
• Add ExDatalog.Constraints.BeamCallback with apply_callback/3; spawn_monitor-based isolation (not Task.async) - closed Add ExDatalog.Constraints.BeamCallback with apply_callback/3; spawn_monitor-based isolation (not Task.async) #144
• Add Constraint.Context opts: [] field; threaded from Engine.Naive.do_evaluate_inner - closed Add Constraint.Context opts: [] field; threaded from Engine.Naive.do_evaluate_inner #145
• Add Capabilities aggregate_constraints: boolean, beam_callbacks: boolean - closed Add Capabilities aggregate_constraints: boolean, beam_callbacks: boolean #146
• Add Schema.PredicateMeta struct; predicate/5 macro; register_predicate/6 - closed Add Schema.PredicateMeta struct; predicate/5 macro; __register_predicate__/6 #147
• Add resolve_positive_or_callback/2 in build_program; converts matching body atoms to callbacks - closed Add resolve_positive_or_callback/2 in __build_program__; converts matching body atoms to callbacks #148
• Add Validator.check_callback/2 (module/function/arity), Safety.check_callback_inputs/2 (args must be bound) - closed Add Validator.check_callback/2 (module/function/arity), Safety.check_callback_inputs/2 (args must be bound) #149
• Evaluator: apply_callbacks/3 in normal and aggregate paths; apply_callback_chain/2 + step_callback/1 - closed Evaluator: apply_callbacks/3 in normal and aggregate paths; apply_callback_chain/2 + step_callback/1 #150
• Program.validate_body/1 accepts {:callback, %Callback{}}; body_relations/1 ignores callbacks - closed Program.validate_body/1 accepts {:callback, %Callback{}}; body_relations/1 ignores callbacks #151
• Tests: beam_callback_test.exs - closed Tests: beam_callback_test.exs #152

M4 — Magic Sets (Experimental) - closed #153

• Add ExDatalog.MagicSets: transform/2 → {:ok, ir} | {:fallback, reason} - closed Add ExDatalog.MagicSets: transform/2 → {:ok, ir} | {:fallback, reason} #154
• Scope: positive recursive programs, single goal, ground bound positions; negation/aggregates → fallback - closed Scope: positive recursive programs, single goal, ground bound positions; negation/aggregates → fallback #155
• Transform: adornment → magic relation + seed fact → rewrite goal-relation rules (prepend magic atom for bound head positions) - closed Transform: adornment → magic relation + seed fact → rewrite goal-relation rules (prepend magic atom for bound head positions) #156
• bound_positions/1 maps goal bound positions to head term positions - closed bound_positions/1 maps goal bound positions to head term positions #157
• Verified: magic restricts derivation and equals semi-naive subset - closed Verified: magic restricts derivation and equals semi-naive subset #158
• Tests: magic_sets_test.exs + magic_sets_property_test.exs - closed Tests: magic_sets_test.exs + magic_sets_property_test.exs #159

M5 — Docs, Benchmarks, CHANGELOG, Version Bump - closed #160

• Version bumped 0.4.1 → 0.5.0 in mix.exs
• Added {:benchee, "~> 1.3", only: :dev} dependency
• Added planner/magic-sets/callback module groups to mix.exs docs
• Added articles 06–09 and migration_v0.5.md as extras
• CHANGELOG.md: full v0.5.0 entry (Added/Changed/Notes)
• README.md: aggregate/callback/magic-sets sections, updated feature list, architecture diagram, constraints table, docs links, roadmap
• Articles: 06_query_planning_in_datalog.md, 07_aggregates_in_datalog.md, 08_extending_datalog_with_beam_callbacks.md, 09_magic_sets_and_demand_driven_evaluation.md
• Migration guide: docs/migration_v0.5.md
• Livebook: livebooks/ex_datalog_v050.livemd (DSL edition)
• Benchmarks: bench/aggregate_bench.exs, bench/magic_sets_bench.exs
• Report: chest/benchmarks/v0.5.0-report.md

Runtime Facts API (pipe-friendly Schema → Program flow)

• Schema relation constructors: each relation generates a public function (e.g., DeptCount.emp(:alice, :eng) → {"emp", [:alice, :eng]})
• Schema.new/0: returns blank program with relations + rules but no facts
• Program.add_fact/2 (tuple form): Program.add_fact(prog, {"emp", [:alice, :eng]})
• Program.add_facts/2: bulk addition from list of {relation, values} tuples
• Program.materialize/1,2: pipe-friendly ExDatalog.materialize/2 wrapper
• Schema.program/0 and materialize/1 unchanged for backward compatibility