Member Avatar for ravenous

At work we have a large number of unit tests that try to alert us that we have made changes that have broken our code. Many of these tests work by evaluating functions under known conditions and comparing the result with the known result. The known result is acquired by manually testing the newly written function with a number of conditions that should exercise all the features of it. We get the output value of the function from this test and then use that as a reference value in the unit test. Very often, a function returns a floating point value ( double , usually).

This is all fine, but occasionally the changes that you makes to the code will pass all the unit tests on your computer, but then later fail on the automatic testing machine that tests all builds once they're committed to the source control repository. This type of failure often results from a tiny difference between the reference value and the value a function has returned. The differences are really small, like in the 12th or greater decimal place. So, my actual question is: could these differences be due to differences in floating point calculations between processors/other hardware components?. If so, does anyone know how these differences can arise?

For information, all code is written in C++ and compiled with MS Visual Studio 2010 (at least on the local machines and I imagine on the test machine too).

  • 4 Contributors
  • 4 Replies
  • 325 Views
  • 1 Year Discussion Span
  • Latest Post Latest Post by santakdalai90

Dani AI

Generated

Small differences in the last few decimal places are normal and not necessarily a bug in your algorithm. They can come from ordinary rounding noise plus a handful of reproducibility traps: different FP hardware (the x87 unit historically uses an 80‑bit extended internal format while SSE/SSE2 uses 64‑bit doubles), different compiler optimizations that change expression ordering, fused multiply‑add (FMA) contraction, or different math‑library implementations of functions like sin/pow. (en.wikipedia.org)

The compiler and runtime can make those tiny differences bigger or smaller. MSVC (and other compilers) provide floating‑point models (/fp:precise, /fp:fast, /fp:strict) and can enable/disable contractions (FMA); changing those changes results and performance. If the build machine uses different defaults than your dev box you can get mismatches. Also the CRT/libm implementations are not required to be identically rounded on every platform. (learn.microsoft.com)

Practical steps to make unit tests stable:

  • Stop using raw == (as suggested). Use a scaled relative tolerance or an ULP test instead of plain machine epsilon. For most cases a relative epsilon plus a tiny absolute floor works well; for tight numeric code use an ULP comparator (counts representable steps between values). Bruce Dawson’s writeup shows good, battle‑tested patterns. (randomascii.wordpress.com)

Example (illustrative — adapt tolerances for your domain):

// relative+absolute tolerance
inline bool almost_equal_rel(double a,double b,double rel=1e-12,double abs_tol=1e-15){
    double diff = std::fabs(a-b);
    if(diff <= abs_tol) return true;
    return diff <= rel * std::max(std::fabs(a), std::fabs(b));
}

// simple ULP-based approach (works well for nearby values)
bool almost_equal_ulps(double a,double b,int64_t maxUlps = 4){
    if(std::isnan(a) || std::isnan(b)) return false;
    if(a==b) return true;
    int64_t ia, ib; std::memcpy(&ia,&a,sizeof(a)); std::memcpy(&ib,&b,sizeof(b));
    if(ia < 0) ia = 0x8000000000000000LL - ia;
    if(ib < 0) ib = 0x8000000000000000LL - ib;
    return llabs(ia-ib) <= maxUlps;
}

For inequality (question from ) use !almost_equal(...) rather than plain !=.

If you require bit‑for‑bit reproducibility: build and run tests on the same target CPU and toolchain, force a reproducible FP model (e.g. set /fp:precise or /fp:strict, disable contraction), and ensure the same runtime and MXCSR/FTZ settings are used on all machines. If that isn’t feasible, make tests tolerant (relative/ULP) or compute references with higher precision (MPFR) so comparisons are meaningful. (learn.microsoft.com)

References: primer on FP pitfalls and techniques by Goldberg; practical comparator guidance by Bruce Dawson; MSVC floating‑point model and contraction docs. (docs.oracle.com)

Recommended Answers

All 4 Replies

Member Avatar for triumphost

I would assume these slight differences would be due to rounding... as we know floats and doubles never ever come out pefectly and is usually rounded. Not sure if machines have anything to do with it though but there is a possibility that it does.. I wouldnt bet my money on the difference in processors.. I dont think a sandy bridge processor should affect the result in comparison to the i7.. I just believed those were rounding tolerances :S

Member Avatar for template<>

Comparing floating point numbers using == or != is not safe. The difference between the two numbers must be compared to some tolerance, usually the system epsilon. (see sample code below).

bool isEqual( float v1, float v2 )
{
    if( std::abs(v1-v2) < std::numeric_limits<float>::epsilon() )
    {
        return true;
    }
    
    return false;
}
commented: Helpful information +6
Member Avatar for ravenous

Comparing floating point numbers using == or != is not safe. The difference between the two numbers must be compared to some tolerance, usually the system epsilon. (see sample code below).

bool isEqual( float v1, float v2 )
{
    if( std::abs(v1-v2) < std::numeric_limits<float>::epsilon() )
    {
        return true;
    }
    
    return false;
}

Good knowledge, I'll definitely be using something like this in the future :) Actually, on my system, epsilon() evaluates to something around 2e-16, so that's much smaller than the differences that I'm seeing, so maybe it's not a floating-point issue after all.

EDIT: You'd think that == would be defined for different types in terms of epsilon() . I guess it would slow down comparisons if it was implemented this way.

Edited by ravenous because: n/a
Member Avatar for santakdalai90

When you are considering epsilon closeness as a condition for equality, how would you then compare for inequality?

should I code like:

bool isNotEqual(float A, float B){
    if(A != B)
        return true;
    else
        return false;
}

or should I code like:

bool isEqual( float v1, float v2 )
{
    if( fabs(v1-v2) > std::numeric_limits<float>::epsilon() ){
        return true;
    }
        return false;
}
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.