Floating Point Precision

Floating point precision in Rust

Part 1 - Part 2

Storing floating point numbers on a computer is a compromise. The following program gives you an idea of the accuracy of 32bit and 64bit precision when using Rust.

The Rust code

fn main() {
    // Initialize floating point numbers
    let mut s23: f32 = 2.0 / 3.0; // Single precision (32-bit)
    let mut d23: f64 = 2.0 / 3.0; // Double precision (64-bit)

    // Perform the first series of operations
    println!("Performing operations with initial values of 2/3");
    println!("{:<20} {:<20}", "s23 (f32)", "d23 (f64)");
    println!("{:<20} {:<20}", "---------", "---------");
    for _ in 1..=18 {
        s23 = s23 / 10.0 + 1.0;
        d23 = d23 / 10.0 + 1.0;
        println!("{:<20} {:<20}", s23, d23);

    // Perform the second series of operations
    println!("\nRepeating operations in reverse with values carried over from last loop:");
    println!("{:<20} {:<20}", "s23 (f32)", "d23 (f64)");
    println!("{:<20} {:<20}", "---------", "---------");
    for _ in 1..=18 {
        s23 = (s23 - 1.0) * 10.0;
        d23 = (d23 - 1.0) * 10.0;
        println!("{:<20} {:<20}", s23, d23);

The code output

Performing operations with initial values of 2/3:
s23 (f32)            d23 (f64)           
---------            ---------           
1.0666667            1.0666666666666667  
1.1066667            1.1066666666666667  
1.1106666            1.1106666666666667  
1.1110667            1.1110666666666666  
1.1111066            1.1111066666666667  
1.1111107            1.1111106666666666  
1.111111             1.1111110666666666  
1.1111112            1.1111111066666666  
1.1111112            1.1111111106666667  
1.1111112            1.1111111110666667  
1.1111112            1.1111111111066667  
1.1111112            1.1111111111106666  
1.1111112            1.1111111111110668  
1.1111112            1.1111111111111067  
1.1111112            1.1111111111111107  
1.1111112            1.1111111111111112  
1.1111112            1.1111111111111112  
1.1111112            1.1111111111111112  

Repeating operations in reverse with values carried over from last loop:
s23 (f32)            d23 (f64)           
---------            ---------           
1.1111116            1.1111111111111116  
1.1111164            1.111111111111116   
1.1111641            1.1111111111111605  
1.1116409            1.1111111111116045  
1.1164093            1.1111111111160454  
1.164093             1.1111111111604544  
1.6409302            1.1111111116045436  
6.4093018            1.1111111160454357  
54.093018            1.1111111604543567  
530.9302             1.1111116045435665  
5299.302             1.111116045435665   
52983.016            1.11116045435665    
529820.1             1.1116045435665     
5298191              1.1160454356650007  
52981900             1.160454356650007   
529819000            1.6045435665000696  
5298190300           6.045435665000696   
52981903000          50.45435665000696 

Mission Accomplished

But what are we seeing?

The output from the Rust program above provides a compelling illustration of the limitations and inherent inaccuracies of floating-point arithmetic in computer programming. Analysing the output, especially in the context of high-level computer science and numerical analysis, sheds light on several key concepts, such as precision, rounding errors, and numerical stability.

Floating-Point Precision and Rounding Errors

The two types of floating-point numbers used here, f32 and f64, represent single and double precision IEEE 754 standards, respectively. f32 provides approximately 7 decimal digits of precision, while f64 offers about 15. This difference in precision is evident from the output.

Initially, both s23 (f32) and d23 (f64) are set to 2/3, but the representation of this value already differs slightly between the two due to their differing precisions. As the operations progress, these small discrepancies become more pronounced. This is a classic demonstration of rounding errors, which occur because floating-point numbers cannot exactly represent all real numbers and must round to the nearest representable value.

Propagation of Errors in Sequential Operations

The program performs a series of additions and multiplications on these numbers. In floating-point arithmetic, errors can propagate and even amplify through such operations. The first loop adds 1 and then divides the result by 10 repeatedly. With each iteration, the impact of the initial rounding error becomes more pronounced, especially for f32. This effect is a manifestation of the error propagation characteristic of floating-point operations.

In the second loop, the program reverses the operations without resetting the values, multiplying by 10 and then subtracting 1. This reversal amplifies the errors even further. As the values are scaled up, the imprecision inherent in the floating-point representation leads to increasingly significant deviations from the expected results.

Numerical Stability and Accumulation of Errors

The concept of numerical stability in algorithms is crucial in understanding this output. An algorithm is numerically stable if an error, once introduced, does not significantly grow as the computation proceeds. The operations in this program, particularly the repetitive scaling and accumulation in the loops, are not numerically stable for floating-point arithmetic. This lack of stability leads to the accumulation of errors over iterations.

For s23 (f32), the errors become substantial quite quickly, illustrating the limitations of single-precision floating points in maintaining accuracy over sequences of operations. The d23 (f64) values remain more accurate for a longer sequence of operations but eventually also deviate significantly, underscoring that while higher precision can mitigate the rate of error accumulation, it cannot eliminate it.

Implications in High-Level Computing and Numerical Analysis

This program is a practical reminder of the challenges faced in computational tasks requiring high precision, such as scientific computations, financial algorithms, and graphics programming. It underscores the importance of choosing the appropriate data type based on the required precision and the nature of the computations. Moreover, it highlights the need for numerically stable algorithms, especially in fields where accuracy is paramount.

The output from this Rust program vividly demonstrates the inherent inaccuracies of floating-point arithmetic due to limited precision, rounding errors, and error propagation. It serves as a cautionary tale for programmers and computer scientists, emphasizing the need for careful selection of data types and algorithms, particularly in applications where precision is critical.

Part 1 - Part 2