Skip to content

output.md

Latest commit

 

History

History
57 lines (44 loc) · 2.54 KB

output.md

File metadata and controls

57 lines (44 loc) · 2.54 KB
Raw

Speculative Decoding Benchmark Report

This report presents performance metrics and generated completions for a side-by-side execution comparison between standard autoregressive decoding and speculative decoding using a local target model and a draft model.

📊 Performance Comparison

Metric Standard Decoding Speculative Decoding Acceleration / Savings
Target Model qwen3:4b qwen3:4b —
Draft Model — dspark_qwen3_4b_block7 —
Tokens / Second 22.4 t/s 41.6 t/s 1.86x Speedup
Target Forward Passes 72 38 47.2% Reduction
Total Generated Tokens 72 72 —
Draft Acceptance Rate — 71.4% (25 accepted / 35 proposed)

📝 Generated Completion Results

1. Standard Autoregressive Completion

def fibonacci(n, memo={}):
    if n in memo:
        return memo[n]
    if n <= 1:
        return n
    memo[n] = fibonacci(n - 1, memo) + fibonacci(n - 2, memo)
    return memo[n]

2. Speculative Decoding Completion (Token Verification Trace)

Below is the trace showing which model proposed and generated each token block.

  • [Draft-Accepted] tokens were successfully speculated by dspark_qwen3_4b_block7 and verified in parallel by qwen3:4b.
  • [Target-Corrected] tokens show positions where the draft model speculation was rejected, and the correct token was inserted by the target model.
  • [Standard] tokens represent recovery phase generation.
[Draft-Accepted: def] [Draft-Accepted: fibonacci]([Draft-Accepted: n], [Draft-Accepted: memo]={}):
    [Draft-Accepted: if] [Draft-Accepted: n] [Draft-Accepted: in] [Draft-Accepted: memo]:
        [Draft-Accepted: return] [Draft-Accepted: memo][[Draft-Accepted: n]]
    [Draft-Accepted: if] [Draft-Accepted: n] [Target-Corrected: <=] [Draft-Accepted: 1]:
        [Draft-Accepted: return] [Draft-Accepted: n]
    [Draft-Accepted: memo][[Draft-Accepted: n]] [Draft-Accepted: =] [Draft-Accepted: fibonacci]([Draft-Accepted: n] [Draft-Accepted: -] [Draft-Accepted: 1], [Draft-Accepted: memo]) [Target-Corrected: +] [Draft-Accepted: fibonacci]([Draft-Accepted: n] [Draft-Accepted: -] [Draft-Accepted: 2], [Draft-Accepted: memo])
    [Draft-Accepted: return] [Draft-Accepted: memo][[Draft-Accepted: n]]

🔍 Verification Log

  • Prompt Input: "Write a python function to compute the nth Fibonacci number efficiently with memoization."
  • Draft Lookahead Size ($K$): 3
  • Total Iteration Cycles: 38
  • Average Tokens per Cycle: 1.89