Hazards of Real Numbers (Advanced)

Calculations with real numbers must be done carefully. Unlike integers, there are many instances where a real number is only an approximation of the desired value. For example, just as the value 1/3 cannot be exactly represented by a decimal number (only approximated by 0.333333333333...), it also cannot be exactly represented by an XPL0 real number. The discrepancy is called a rounding error. A real must round the true value to the nearest value it can represent.

Because of rounding errors an expression like:

        9.0 * (1.0 / 3.0)

does not evaluate to exactly 3.0. The intermediate result, 0.33333333333, is not 1/3, and 0.3333333333333333 times 9.0 is 2.9999999999999997. Yet if the order of this calculation is changed, the result is exactly 3.0:

        (9.0 * 1.0) / 3.0

These two expressions are not exactly equal. Thus the first hazard of real numbers is testing for equality. Usually, it is only a coincidence if a real expression evaluates to an exact value. This problem is obscured because if we were to output the values of the two preceding expressions using the RlOut intrinsic, we would get 3.000000000000000000 in both cases. The reason is RlOut itself rounds to compensate for slight rounding errors.

The second hazard of rounding errors is that they can accumulate to cause big errors. For example, if an expression such as:

        3.0 * (1.0 / 3.0)

is multiplied by itself 1000 times, the result might be something like 1.000000000000220.

Another hazard to be wary of is loss of accuracy caused by subtracting. For example, the expression

     1234567890123456. - 1234567890123454. + 1.25

equals 3.25, but the same expression evaluated in a different order

     1234567890123456. - (1234567890123454. + 1.25)

equals 1.0.

The discrepancy is caused by not having more than 16 digits of accuracy. When 1234567890123454 is added to 1.25, the result is rounded to 1234567890123455. This discrepancy can be seen two ways. Certainly the difference between 3.25 and 1.0 seems significant, but 2.25 compared to 1234567890123456 is really quite small.