Type Confusion: Positive NaN is misinterpreted as T_CODEREF

[hkbinbin]

Summary

A type confusion vulnerability exists in Elk's NaN-boxing implementation where IEEE 754 positive NaN values (0x7ff8000000000000) are incorrectly interpreted as the internal T_CODEREF type instead of being treated as invalid/error values.

Root Cause

The is_nan() function only checks for positive NaN/Infinity patterns:

// elk.c line 139
static bool is_nan(jsval_t v) { 
    return (v >> 52U) == 0x7ffU;  // Only checks positive NaN!
}

This causes negative NaN/Infinity (0xfff...) to be treated as regular T_NUM, allowing arithmetic operations that can produce positive NaN.

Proof of Concept

// Step 1: Create negative infinity (passes T_NUM check, can do arithmetic)
let ninf = 0 - 1e308 - 1e308;

// Step 2: Create negative NaN via -inf - -inf (still T_NUM)
let nan = ninf - ninf;

// Step 3: Negate to get positive NaN (becomes T_CODEREF!)
let pnan = -nan;

// Step 4: Verify type confusion
typeof pnan  // Returns "coderef" instead of "number"!

Expected Behavior

typeof pnan should return "number" or the operation should produce an error.

Actual Behavior

typeof pnan returns "coderef", indicating the value has been misinterpreted as an internal code reference type.

Technical Analysis

Exploitation Path

0 - 1e308 - 1e308 = -Infinity (0xfff0...)
    │                  vtype = 5 (T_NUM) ← passes arithmetic check
    ▼
(-inf) - (-inf) = -NaN (0xfff8...)
    │                  vtype = 5 (T_NUM) ← still passes check
    ▼
-(-NaN) = +NaN (0x7ff8...)
                       vtype = 8 (T_CODEREF) ← TYPE CONFUSION!

Why This Happens

1. is_nan() only recognizes positive NaN patterns (0x7ff...)
2. Negative infinity/NaN (0xfff...) bypasses the check → treated as T_NUM
3. Arithmetic on negative infinity produces negative NaN
4. Negating negative NaN flips the sign bit → positive NaN
5. Positive NaN's bits 51:48 equal 8 → interpreted as T_CODEREF

Impact

The positive NaN value can:

• Be stored in variables and object properties
• Be passed as function arguments
• Be returned from functions
• Pass typeof checks as "coderef"

Suggested Fix

Fix is_nan() to detect both positive and negative NaN

static bool is_nan(jsval_t v) { 
    return ((v >> 52U) & 0x7ffU) == 0x7ffU;  // Ignore sign bit
}