Fix overflow and underflow in trust region lengthscale normalization

[gian21391]

Problem

TurboController.get_trust_region scales the trust region by the model's lengthscales:

weights = lengthscales / torch.prod(lengthscales) ** (1 / active_dim)

torch.prod(lengthscales) multiplies together one number per dimension. With enough dimensions that product runs out of floating-point range — and it can go wrong in both directions:

• Large lengthscales — the product overflows to inf, every weight becomes 0, and the trust region shrinks to a single point.
• Small lengthscales — the product underflows to 0, every weight becomes inf, and the trust region gets clamped out to the whole domain.

Either way there is no warning, no NaN and no exception. The optimizer just quietly stops behaving like TuRBO.

Overflow needs unusually large lengthscales (degenerate GP fits). The product exceeds the dtype maximum once the geometric mean passes max ** (1 / d):

dtype	dims	overflows above
float64	32	~4.3e9
float64	100	~1.2e3
float32	40	~9.2

So in float32 a lengthscale of 12 at d=40 already does it.

Underflow needs nothing unusual — just a lot of dimensions. 0.3 ** 700 is already zero in float64, and lengthscales below 1 are normal for normalized inputs.

Fix

Take the root of each lengthscale before multiplying:

weights = lengthscales / torch.prod(lengthscales ** (1 / active_dim))

Mathematically the same value, but every factor is now close to 1, so the product stays in range. This is the form the BoTorch BAxUS tutorial uses.

Test

test_get_trust_region_extreme_lengthscales now covers both cases, 1e200 and 1e-200. Both fail on main and pass with the fix:

[overflow]   region collapsed to a point, widths [0.0, 0.0]
[underflow]  region blown out past the trust region, widths [1.0, 10.0] exceed [0.25, 2.5]

It checks 0 < width <= length * bound_width rather than an exact width, so it still passes when the incumbent sits near a bound and the region is legitimately clipped.

The test is a module-level function rather than a TestTurbo method because pytest cannot parametrize unittest.TestCase methods. Existing turbo and contextual-BO tests are unchanged.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!