Floating-Point Comparison Pitfalls

Comparing floating-point numbers for equality is one of the most common sources of bugs in software. The fundamental problem is that many exact decimal values cannot be stored exactly in binary floating-point, and arithmetic operations introduce rounding errors.

The naive comparison trap:

// This is almost always wrong:
if (a === b) { ... }

// This is also problematic for calculated values:
if (result === 0.3) { ... }

Method 1: Absolute epsilon

function nearlyEqual(a, b, epsilon = 1e-10) {
  return Math.abs(a - b) < epsilon;
}

This works for values near zero but fails for large values. At x = 1e20, the gap between adjacent floats is about 16384, so an epsilon of 1e-10 would never match two values that should be considered equal.

Method 2: Relative epsilon

function nearlyEqual(a, b, tolerance = Number.EPSILON * 10) {
  const diff = Math.abs(a - b);
  const norm = Math.max(Math.abs(a), Math.abs(b));
  return diff < norm * tolerance;
}

This scales with the magnitude of the values. However, it fails near zero because the relative error becomes infinite.

Method 3: Combined approach

function nearlyEqual(a, b, relTol = 1e-9, absTol = 1e-12) {
  const diff = Math.abs(a - b);
  if (diff < absTol) return true; // handles near-zero case
  const norm = Math.max(Math.abs(a), Math.abs(b));
  return diff < norm * relTol;
}

This is the approach used by Python's math.isclose() and NumPy's numpy.allclose().

Method 4: ULP-based comparison

Compare how many representable floats apart two values are:

function ulpDistance(a, b) {
  const buf = new ArrayBuffer(8);
  const f64 = new Float64Array(buf);
  const i64 = new BigInt64Array(buf);
  f64[0] = a; const ai = i64[0];
  f64[0] = b; const bi = i64[0];
  return Number(ai > bi ? ai - bi : bi - ai);
}

Two values within 1-4 ULPs are typically considered equal for single arithmetic operations.

Special cases to remember:

• NaN !== NaN (NaN is never equal to anything, including itself)
• +0 === -0 (but they are distinguishable with Object.is)
• Infinity === Infinity (infinity does compare equal to itself)
• Subtraction of nearly equal values causes catastrophic cancellation