[SCCP] Fold y = f(x) = (Cx mod M) ∈ R into x ∈ R' - #186347
Conversation
ba8036a to
5280b33
Compare
|
@llvm/pr-subscribers-llvm-transforms Author: Kunqiu Chen (Camsyn) ChangesThis PR introduces an optimization for a very common pattern in integer code (e.g., %y = (mul (zext %x), C)
icmp pred (%y + C1), C2If the multiplication is invertible under the predicate range, the comparison can be rewritten into a direct comparison on This enables the following transformation: %y = (mul (zext %x), C)
icmp pred (%y + C1), C2
-->
icmp pred (%x + C3), C4Notably, we can extend this optimization to the case where Currently, for simplicity, this patch does not consider E.g., consider the following LLVM IR: %ext = zext i4 %x to i8
%mul = mul i8 %ext, 20
%cmp = icmp sge i8 %mul, 60 ; predicate range: [60, 128)The predicate constrains the result of the multiplication: Because the multiplication is linear and invertible in this region, we can map the constraint back to $$ Thus the condition is equivalent to: %sub = sub i4 %x, 3
%cmp = icmp ult i4 %sub, 4which removes the multiplication while preserving semantics. Mathematical ModelWe model the multiplication as a modular map $$ over the domain $$ corresponding to LLVM integer types As Viewed continuously, the mapping is approximately periodic with period $$ If $$ Considering integer multiplication constants satisfy $$ The optimization $$ Here is a figure summarizing the mathematical model. <img width="1800" height="800" alt="image" src="https://github.com/user-attachments/assets/dfeeaf32-0c62-4d0f-ab36-fd6dac11ff31" />
Full diff: https://github.com/llvm/llvm-project/pull/186347.diff 2 Files Affected:
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineCompares.cpp b/llvm/lib/Transforms/InstCombine/InstCombineCompares.cpp
index 7f1ced9505b9b..8ea28e8fce3da 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineCompares.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineCompares.cpp
@@ -32,6 +32,7 @@
#include "llvm/Support/KnownBits.h"
#include "llvm/Transforms/InstCombine/InstCombiner.h"
#include <bitset>
+#include <optional>
using namespace llvm;
using namespace PatternMatch;
@@ -2178,7 +2179,165 @@ Instruction *InstCombinerImpl::foldICmpOrConstant(ICmpInst &Cmp,
return nullptr;
}
-/// Fold icmp (mul X, Y), C.
+/// Refer to https://github.com/llvm/llvm-project/pull/186347 for the
+/// underlying math model.
+///
+/// Compute the result range Y on which y = Cx mod m is invertible, where
+/// x ranges over [0, n - 1], n = 2^N, and m = 2^M.
+///
+/// When IsDomainReturned is true, the returned Y is also the reachable image of
+/// the multiply, i.e. the domain. Otherwise, the returned Y is only the unique
+/// invertible result range.
+static std::optional<ConstantRange>
+getInvertibleResultRangeForZExtMul(const APInt &C, unsigned N,
+ bool &IsDomainReturned) {
+ unsigned M = C.getBitWidth();
+ assert(N < M && "Expected zext to a wider type");
+
+ // Use a widened type so that n = 2^N, m = 2^M, and (n - 1) * C are all
+ // representable without wrapping during the analysis.
+ unsigned WideBits = N + M;
+ APInt WideC = C.zext(WideBits);
+ APInt n = APInt::getOneBitSet(WideBits, N);
+ APInt m = APInt::getOneBitSet(WideBits, M);
+
+ // k = floor(((n - 1) * C) / m) counts how many times the walk y = Cx crosses
+ // the modulus while x ranges over [0, n - 1].
+ APInt MaxY = (n - 1) * WideC;
+ APInt k = MaxY.lshr(M);
+
+ if (k.isZero()) {
+ // k = 0: the walk never wraps, so Y is the full result space. The compare
+ // still needs to be intersected with the reachable image [0, MaxY].
+ // f(x) = Cx: |
+ // Y | /
+ // Y | /
+ // Y |/
+ // ----
+ IsDomainReturned = true;
+ return ConstantRange::getNonEmpty(APInt::getZero(M), (MaxY + 1).trunc(M));
+ }
+
+ if (k.isOne()) {
+ // k = 1: only the unique upper tail [((n - 1) * C + 1) mod m, m) is
+ // invertible.
+ // f(x) = Cx: Y | /
+ // Y | /
+ // | / /
+ // |/ /
+ // -------
+ IsDomainReturned = false;
+ APInt TailLo = (MaxY - m + 1).trunc(M);
+ return ConstantRange::getNonEmpty(TailLo, APInt::getZero(M));
+ }
+
+ // k >= 2: the walk overlaps itself too much to have a unique inverse.
+ // f(x) = Cx: | / /
+ // | / /
+ // Y does not exist. | / / /
+ // |/ / /
+ // -----------
+ return std::nullopt;
+}
+
+/// Given a compared result range CmpCR, constant C, and number N, considering
+/// y = f(x) = Cx, where C: iM, y: iM, x: iN,
+/// this function tries to find the equivalent range X of x,
+/// where x ∈ X iff y ∈ CmpCR.
+/// Refer to https://github.com/llvm/llvm-project/pull/186347 for the
+/// underlying math model.
+static std::optional<ConstantRange>
+getEquivalentRangeForZExtMul(const ConstantRange &CmpCR, const APInt &C,
+ unsigned N) {
+ assert(N < C.getBitWidth() && "Expected zext to a wider type");
+ assert(!CmpCR.isEmptySet() && "Unexpected empty set");
+ assert(!CmpCR.isFullSet() && "Unexpected full set");
+ assert(!C.isOne() && "mul x, 1 should be folded before.");
+
+ // ==================================================================== //
+ // 1. Calculate the invertible interval Y for mul (zext x), C
+ // ==================================================================== //
+ bool IsDomainReturned = false;
+ auto Y = getInvertibleResultRangeForZExtMul(C, N, IsDomainReturned);
+ if (!Y)
+ return std::nullopt;
+
+ // ==================================================================== //
+ // 2. Calculate the equivalent range X via f^{-1} on CmpCR
+ // ==================================================================== //
+
+ auto TryGetSourceRange =
+ [&](const ConstantRange &CR) -> std::optional<ConstantRange> {
+ // If Y is also the domain, only CR ∩ Y matters. Otherwise, the whole
+ // compare range must stay inside the unique invertible tail.
+ std::optional<ConstantRange> ActiveCmpY;
+ if (IsDomainReturned) {
+ // Y is domain of Cx: keep only the reachable part
+ // ActiveCmpY = null if
+ // L-------U : Y
+ // --U L----- : CR
+ ActiveCmpY = Y->exactIntersectWith(CR);
+
+ if (ActiveCmpY == Y)
+ return /* Y ⊆ CR*/ ConstantRange::getFull(N);
+ if (ActiveCmpY->isEmptySet())
+ return /* Y ∩ CR = ∅ */ ConstantRange::getEmpty(N);
+ } else {
+ // Otherwise, CR must stays entirely inside Y
+ // ActiveCmpY = null of CR has some values non-invertible,
+ ActiveCmpY = Y->contains(CR) ? std::optional(CR) : std::nullopt;
+ }
+ // If ActiveCmpY = null, there are >1 separate intervals of x,
+ // making Cx ∈ CR. I.e., we cannot derive a single X.
+ if (!ActiveCmpY)
+ return std::nullopt;
+ // For an invertible half-open interval Y = [y0, y1), the corresponding
+ // source interval is f^{-1}(Y) = [ceil(y0 / C), ceil(y1 / C)) = [x0, x1).
+ APInt Y0 = ActiveCmpY->getLower();
+ APInt Y1 = ActiveCmpY->getUpper() - 1;
+ // x0 = ceil(y0 / C)
+ APInt X0 = APIntOps::RoundingUDiv(Y0, C, APInt::Rounding::UP);
+ // x1 = ceil(y1 / C) = floor((y1 - 1) / C) + 1
+ APInt X1 = APIntOps::RoundingUDiv(Y1, C, APInt::Rounding::DOWN) + 1;
+ return ConstantRange::getNonEmpty(X0.trunc(N), X1.trunc(N));
+ };
+
+ // Try to get single X to make Cx ∈ CmpCR by f^{-1}(CmpCR)
+ if (auto SrcCR = TryGetSourceRange(CmpCR))
+ return SrcCR;
+ // Try to get single X to make Cx ∈ CmpCR by f^{-1}(CmpCR.inverse()).inverse()
+ if (auto InvSrcCR = TryGetSourceRange(CmpCR.inverse()))
+ return InvSrcCR->inverse();
+
+ return std::nullopt;
+}
+
+/// Materialize the source range returned by getEquivalentRangeForZExtMul() as
+/// an icmp (or a constant true/false) over X.
+static Value *emitICmpForEquivalentRange(Value *X, const ConstantRange &SrcCR,
+ InstCombiner::BuilderTy &Builder) {
+ auto *Ty = cast<IntegerType>(X->getType());
+
+ if (SrcCR.isEmptySet())
+ return Builder.getFalse();
+ if (SrcCR.isFullSet())
+ return Builder.getTrue();
+
+ ICmpInst::Predicate NewPred;
+ APInt NewC, Offset;
+ SrcCR.getEquivalentICmp(NewPred, NewC, Offset);
+
+ Value *NewX = X;
+ if (!Offset.isZero()) {
+ if (Offset.isNegative())
+ NewX = Builder.CreateSub(X, ConstantInt::get(Ty, -Offset));
+ else
+ NewX = Builder.CreateAdd(X, ConstantInt::get(Ty, Offset));
+ }
+
+ return Builder.CreateICmp(NewPred, NewX, ConstantInt::get(Ty, NewC));
+}
+
Instruction *InstCombinerImpl::foldICmpMulConstant(ICmpInst &Cmp,
BinaryOperator *Mul,
const APInt &C) {
@@ -2197,6 +2356,26 @@ Instruction *InstCombinerImpl::foldICmpMulConstant(ICmpInst &Cmp,
if (!match(Mul->getOperand(1), m_APInt(MulC)))
return nullptr;
+ // Try to match and optimize the follow pattern
+ // y = mul (zext x), C
+ // icmp pred y, C2 ; y in CR?
+ // -->
+ // icmp pred‘ (x + C3), C4 ; x in CR'?
+ // if y = Cx is invertible on y \in CR
+ //
+ // This currently only handles scalar, C > 0, and zext-based patterns;
+ // FIXME: can vector, C < 0, and sext-based patterns be supported?
+ Value *NarrowX;
+ if (ICmpInst::isRelational(Pred) && MulTy->isIntegerTy() &&
+ MulC->isStrictlyPositive() && match(X, m_ZExt(m_Value(NarrowX)))) {
+ auto *SrcTy = cast<IntegerType>(NarrowX->getType());
+ ConstantRange CmpCR = ConstantRange::makeExactICmpRegion(Pred, C);
+ if (auto SrcCR =
+ getEquivalentRangeForZExtMul(CmpCR, *MulC, SrcTy->getBitWidth()))
+ return replaceInstUsesWith(
+ Cmp, emitICmpForEquivalentRange(NarrowX, *SrcCR, Builder));
+ }
+
// If this is a test of the sign bit and the multiply is sign-preserving with
// a constant operand, use the multiply LHS operand instead:
// (X * +MulC) < 0 --> X < 0
@@ -3197,6 +3376,27 @@ Instruction *InstCombinerImpl::foldICmpAddConstant(ICmpInst &Cmp,
ConstantInt::get(Ty, C - *C2));
auto CR = ConstantRange::makeExactICmpRegion(Pred, C).subtract(*C2);
+
+ // Try to match and optimize the follow pattern
+ // y = mul (zext x), C
+ // icmp pred (y + C1), C2 ; y in CR?
+ // -->
+ // icmp pred’ (x + C3), C4 ; x in CR'?
+ // if y = Cx is invertible on y \in CR
+ //
+ // This currently only handles scalar, C > 0, and zext-based patterns;
+ // FIXME: can vector, C < 0, and sext-based patterns be supported?
+ Value *NarrowX;
+ const APInt *MulC;
+ if (match(X, m_Mul(m_ZExt(m_Value(NarrowX)), m_APInt(MulC))) &&
+ X->getType()->isIntegerTy() && MulC->isStrictlyPositive()) {
+ auto *SrcTy = cast<IntegerType>(NarrowX->getType());
+ if (auto SrcCR =
+ getEquivalentRangeForZExtMul(CR, *MulC, SrcTy->getBitWidth()))
+ return replaceInstUsesWith(
+ Cmp, emitICmpForEquivalentRange(NarrowX, *SrcCR, Builder));
+ }
+
const APInt &Upper = CR.getUpper();
const APInt &Lower = CR.getLower();
if (Cmp.isSigned()) {
diff --git a/llvm/test/Transforms/InstCombine/icmp-mul.ll b/llvm/test/Transforms/InstCombine/icmp-mul.ll
index 49e1e11fe6c36..d499b38975c69 100644
--- a/llvm/test/Transforms/InstCombine/icmp-mul.ll
+++ b/llvm/test/Transforms/InstCombine/icmp-mul.ll
@@ -1568,3 +1568,280 @@ entry:
%cmp = icmp slt i64 %mul1, %mul2
ret i1 %cmp
}
+
+; Test for icmp (mul (zext x), C) to icmp x,
+; if mul is invertible on given predicate constraint
+; Refer to https://github.com/llvm/llvm-project/pull/186347 to understand
+; the mathematical model.
+
+; Come from https://github.com/llvm/llvm-project/pull/185907#discussion_r2919506475
+; N = 9, M = 27
+; n = 2^9 = 512, m = 2^27 = 134217728, C = 262657
+; k = floor((n - 1) * C / m) = floor(511 * 262657 / 134217728) = 0
+; CR = [-2^26, 262657), Y = [0, 134217728)
+; Invertible: yes
+define i1 @slt_invertible_zext_mul_full_image(<2 x i9> %v) {
+; CHECK-LABEL: @slt_invertible_zext_mul_full_image(
+; CHECK-NEXT: [[E:%.*]] = extractelement <2 x i9> [[V:%.*]], i64 0
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i9 [[E]], 1
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %e = extractelement <2 x i9> %v, i64 0
+ %z = zext i9 %e to i27
+ %m = mul i27 %z, 262657
+ %cmp = icmp slt i27 %m, 262657
+ ret i1 %cmp
+}
+
+; N = 8, M = 16
+; n = 2^8 = 256, m = 2^16 = 65536, C = 257
+; k = floor((n - 1) * C / m) = floor(255 * 257 / 65536) = 0
+; CR = [-2^15, 257), Y = [0, 65536)
+; Invertible: yes
+define i1 @slt_invertible_zext_mul_full_image_i16(i8 %v) {
+; CHECK-LABEL: @slt_invertible_zext_mul_full_image_i16(
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i8 [[V:%.*]], 1
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i8 %v to i16
+ %m = mul nuw i16 %z, 257
+ %cmp = icmp slt i16 %m, 257
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 17
+; k = floor((n - 1) * C / m) = floor(15 * 17 / 256) = 0
+; CR = [-2^7, 17), Y = [0, 256)
+; Invertible: yes
+define i1 @slt_invertible_zext_mul_full_image_i8(i4 %v) {
+; CHECK-LABEL: @slt_invertible_zext_mul_full_image_i8(
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i4 [[V:%.*]], 1
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %v to i8
+ %m = mul nuw i8 %z, 17
+ %cmp = icmp slt i8 %m, 17
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 10
+; k = floor((n - 1) * C / m) = floor(15 * 10 / 256) = 0
+; CR = [0, 50), Y = [0, 151)
+; Invertible: yes, because CR ⊆ Y
+define i1 @ult_invertible_zext_mul_partial_image(i4 %x) {
+; CHECK-LABEL: @ult_invertible_zext_mul_partial_image(
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[X:%.*]], 5
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 10
+ %cmp = icmp ult i8 %m, 50
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 10
+; k = floor((n - 1) * C / m) = floor(15 * 10 / 256) = 0
+; CR = [0, 200), Y = [0, 151)
+; Invertible: yes
+define i1 @ult_invertible_zext_mul_all_true(i4 %x) {
+; CHECK-LABEL: @ult_invertible_zext_mul_all_true(
+; CHECK-NEXT: ret i1 true
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 10
+ %cmp = icmp ult i8 %m, 200
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 10
+; k = floor((n - 1) * C / m) = floor(15 * 10 / 256) = 0
+; CR = [200, 256), Y = [0, 151)
+; Invertible: yes
+define i1 @uge_invertible_zext_mul_all_false(i4 %x) {
+; CHECK-LABEL: @uge_invertible_zext_mul_all_false(
+; CHECK-NEXT: ret i1 false
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 10
+ %cmp = icmp uge i8 %m, 200
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CR = [60, 128), Y = [45, 256)
+; Invertible: yes
+define i1 @sge_invertible_tail_of_zext_mul(i4 %x) {
+; CHECK-LABEL: @sge_invertible_tail_of_zext_mul(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], -3
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], 4
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %cmp = icmp sge i8 %m, 60
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CR = [60, 256), Y = [45, 256)
+; Invertible: yes
+define i1 @uge_invertible_tail_of_zext_mul(i4 %x) {
+; CHECK-LABEL: @uge_invertible_tail_of_zext_mul(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], -3
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], -6
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %cmp = icmp uge i8 %m, 60
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 18
+; k = floor((n - 1) * C / m) = floor(15 * 18 / 256) = 1
+; CR = [-2^7, 16), Y = [15, 256)
+; Invertible: yes on Y.inverse() = [16, 2^7)
+define i1 @slt_noninvertible_signed_range_before_tail(i4 %v) {
+; CHECK-LABEL: @slt_noninvertible_signed_range_before_tail(
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i4 [[V:%.*]], 1
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %v to i8
+ %cast = mul nuw i8 %z, 18
+ %cmp = icmp slt i8 %cast, 16
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CR = [0, 45), Y = [45, 256)
+; Invertible: yes on Y.inverse() = [45, 256)
+define i1 @ult_noninvertible_zext_mul_range(i4 %x) {
+; CHECK-LABEL: @ult_noninvertible_zext_mul_range(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], 3
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], 6
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %cmp = icmp ult i8 %m, 45
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 18
+; k = floor((n - 1) * C / m) = floor(15 * 18 / 256) = 1
+; CR = [0, 16), Y = [15, 256)
+; Invertible: yes on Y.inverse() = [16, 256)
+define i1 @ult_noninvertible_zext_mul_before_tail(i4 %v) {
+; CHECK-LABEL: @ult_noninvertible_zext_mul_before_tail(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[V:%.*]], 1
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], 2
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %v to i8
+ %cast = mul i8 %z, 18
+ %cmp = icmp ult i8 %cast, 16
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CR = [-2^7, 60), Y = [45, 256)
+; Invertible: yes on Y.inverse() = [60, 2^7)
+define i1 @slt_noninvertible_crosses_wrap(i4 %v) {
+; CHECK-LABEL: @slt_noninvertible_crosses_wrap(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[V:%.*]], -7
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], -4
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %v to i8
+ %cast = mul i8 %z, 20
+ %cmp = icmp slt i8 %cast, 60
+ ret i1 %cmp
+}
+
+; N = 5, M = 8
+; n = 2^5 = 32, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(31 * 20 / 256) = 2
+; CR = [60, 256), Y = none
+; Invertible: no
+define i1 @uge_noninvertible_multiple_wraps(i5 %x) {
+; CHECK-LABEL: @uge_noninvertible_multiple_wraps(
+; CHECK-NEXT: [[Z:%.*]] = zext i5 [[X:%.*]] to i8
+; CHECK-NEXT: [[M:%.*]] = mul i8 [[Z]], 20
+; CHECK-NEXT: [[CMP:%.*]] = icmp ugt i8 [[M]], 59
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i5 %x to i8
+ %m = mul i8 %z, 20
+ %cmp = icmp uge i8 %m, 60
+ ret i1 %cmp
+}
+
+; Tests for CmpCR built through add.
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CmpCR = [110, 228), Y = [45, 256)
+; Invertible: yes
+define i1 @sge_invertible_tail_of_zext_mul_plus_offset(i4 %x) {
+; CHECK-LABEL: @sge_invertible_tail_of_zext_mul_plus_offset(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], -6
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], 6
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %a = sub i8 %m, 100
+ %cmp = icmp sge i8 %a, 10
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CmpCR = [100, 200), Y = [45, 256)
+; Invertible: yes
+define i1 @ult_invertible_zext_mul_plus_offset(i4 %x) {
+; CHECK-LABEL: @ult_invertible_zext_mul_plus_offset(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], -5
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], 5
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %a = add i8 %m, -100
+ %cmp = icmp ult i8 %a, 100
+ ret i1 %cmp
+}
+
+; N = 4, M = 8
+; n = 2^4 = 16, m = 2^8 = 256, C = 20
+; k = floor((n - 1) * C / m) = floor(15 * 20 / 256) = 1
+; CmpCR = [122, 60), Y = [45, 256)
+; Invertible: yes on CmpCR.inverse() = [60, 122)
+define i1 @slt_inverse_invertible_zext_mul_plus_offset(i4 %x) {
+; CHECK-LABEL: @slt_inverse_invertible_zext_mul_plus_offset(
+; CHECK-NEXT: [[TMP1:%.*]] = add i4 [[X:%.*]], -7
+; CHECK-NEXT: [[CMP:%.*]] = icmp ult i4 [[TMP1]], -4
+; CHECK-NEXT: ret i1 [[CMP]]
+;
+ %z = zext i4 %x to i8
+ %m = mul i8 %z, 20
+ %a = add i8 %m, 6
+ %cmp = icmp slt i8 %a, 66
+ ret i1 %cmp
+}
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
e561315 to
b0c3c32
Compare
91c1e25 to
1680708
Compare
e57ceda to
7c25ba5
Compare
There was a problem hiding this comment.
Pull request overview
This PR extends SCCP’s range-check simplification by folding invertible periodic mappings (modeled as modular multiplication) back onto the preimage domain, enabling comparisons on the original value (x) instead of the mapped value (y = f(x)), and adds an “eager” mode to apply these more aggressive rewrites in the SCCP pass.
Changes:
- Add modular-mapping preimage computation (
f(x) ∈ R→x ∈ f⁻¹(R)) formul/shl/urem/maskedandand integrate it into SCCP’s icmp simplification (guarded by an “Eager” mode). - Extend/adjust SCCP range-check relaxation to prefer single-icmp forms when possible.
- Add new SCCP regression tests covering invertible / non-invertible cases, eager-vs-non-eager behavior, and phase-ordering expectations.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| llvm/lib/Transforms/Utils/SCCPSolver.cpp | Implements periodic-mapping preimage logic and threads an “Eager” knob into icmp simplification / range-check relaxation. |
| llvm/include/llvm/Transforms/Utils/SCCPSolver.h | Adds the Eager parameter to simplifyInstsInBlock() and documents when to use it. |
| llvm/lib/Transforms/Scalar/SCCP.cpp | Enables eager simplification mode for the SCCP pass. |
| llvm/test/Transforms/SCCP/relax-range-checks.ll | Adds new tests for relaxing two-instruction range checks into single-icmp checks (plus a negative test). |
| llvm/test/Transforms/SCCP/invertible-periodic-linear-mapping.ll | New test file covering invertible periodic mapping folds (mul/urem/shl/and + vector splat cases). |
| llvm/test/Transforms/SCCP/eager-invertible-periodic-mapping.ll | New test asserting eager SCCP performs the preimage fold while IPSCCP does not. |
| llvm/test/Transforms/PhaseOrdering/cmp-logic.ll | Updates expected output due to improved simplification (removal of redundant urem). |
This PR introduces an optimization that reverts a range check for a modular multiplication mapping to a range check for the preimage.
If the multiplication is invertible under the predicate range R, the comparison can be rewritten into a direct comparison on
x, eliminating the multiplication entirely.I.e., fold$y = f(x) = (Cx \bmod M) ∈ R$ into $x ∈ R'$ , where $R' = f^{-1}(R)$ .
This enables the following transformation:
Notably, we can extend this optimization to the case where
y = Cxis invertible onR.inverse()(R is the predicate range), i.e.,C < 0, as we can negate the sign byurem/shl/and: all of them can be modeled as a modular multiplication.mul <4 x i8> %vec, splat (i8 50)TODO:
x(e.g.,sext)trunc, which can also be treated as a modular multiplication.srem).Mathematical Model
We model the multiplication as a modular map with$C > 0$
over the UNWRAPPED range$x \in R_x = [x_0, x_1]$ (i.e, $x_1 > x_0$ ).
As$x$ increases, $f(x)$ advances through $[0,M)$ with step size $C$ .
Viewed continuously, the mapping is approximately periodic with period
The number of periods covered is
Considering integer multiplication constants satisfy$C \ge 2$ , three cases arise:
The optimization$y = Cx \in R = [y_0, y_1) \to x \in R' = f^{-1}(R)$ is correct only when the linear inverse $f^{-1}$ exists, i.e., when
Here is a figure summarizing the mathematical model (with$R_x = [0, x_1]$ ).
bitcast [x,...,x] to iNtomul x, C#185907 (comment)