Complexity Analysis

This very short chapter introduces algorithmic complexity and the associated vocabulary. A few examples are presented but this chapter mainly serves as a foundation for the rest of the course (notably for arrays, sorting algorithms…).

Motivation

Given that for a given program or subprogram there are several ways to implement it, and that for some programs (especially in the case of embedded systems) there are strong hardware constraints, it is necessary to evaluate the performance of the algorithms we write. This notion of performance corresponds to algorithmic complexity where we seek to estimate in advance:

the number of operations required to carry out a subprogram: time complexity
the memory space required to carry out this subprogram: space complexity
the usage limits of a subprogram (i.e., knowing when a subprogram is no longer efficient compared to another)

For this, we evaluate the number of elementary operations (assignments, additions, subtractions…, comparisons…), the number of loop iterations, the number of memory cells used… The complexity of a program often depends on input variables (values requested from the user, measured data…). In general, complexity calculation is performed on:

the average of all possible executions of the program
the best case (i.e., the execution that minimizes the quantity being measured)
the worst case (i.e., the execution that maximizes the quantity being measured)

It is also expressed asymptotically, that is, as a limit for large values of the input parameters.

Examples

Constant Complexity

Consider the following procedure for which we seek to measure complexity in terms of tests/comparisons.


void print_max(int a, int b)
{
    int maxi;

    if (a < b)  maxi = b;
    else        maxi = a;
    printf("The maximum of %d and %d is: %d\n", a, b, maxi);
}

Regardless of the values of a and b, there will only be one comparison. Therefore the time complexity (where we count tests) is on average / in the best case / in the worst case equal to 1. The asymptotic expression of this complexity is O(1).

Case Where the Best Scenario Differs from the Worst

Consider the following function for which we seek to measure complexity in terms of number of assignments.


int dummy(int n)
{
    int s = 0;
    if ( n > 0 )
    {
        for (int i = 0; i < n; i++)
        {
            s = s + 1;
        }
    }
    return s;
}

Here we observe that the best case is reached when n is negative or zero since in that case there is only one assignment (the initialization of s). Conversely if n is positive, it is the worst case since there are n+2 assignments (initialization of s then n+1 loop iterations where there is 1 assignment of s).

Asymptotic Complexity

Asymptotic complexity is calculated for subprograms when the value (or the evolution of the value) of the input parameters influences the performance of the algorithm. In the function below, the complexity in assignments and additions is linear (notation O(n)). Indeed, for any n, there are n additions and n+1 assignments. When n becomes very large the n+1 is negligible compared to n and therefore the assignment and addition cost of this function is proportional to n.


int sum(int n)
{
    int sum = 0;
    for (int i = 0; i < n; i++)
    {
        s += i;
    }
    return sum;
}

The most common asymptotic complexities are the following:

O(1): constant, independent of the input parameters
O(n): linear with respect to the input parameters
O(n²): quadratic with respect to the input parameters
O(P(n)) where P is a polynomial in n: polynomial with respect to the input parameters
O(2^n): exponential with respect to the input parameters
O(log(n)): logarithmic with respect to the input parameters

The big-O cheat sheet website provides asymptotic complexities for a large number of common algorithms.

:::info Note The O(x) notation is most commonly used because it allows bounding the maximum execution time of the algorithm, but complexity can be determined more precisely with the following notations:

o(N): complexity strictly less than N
𝜃(N): complexity strictly equal to N
Ω(N): complexity greater than or equal to N
ѡ(N): complexity strictly greater than N :::

As an example, the function below has quadratic complexity (if we count the number of arithmetic operations):


int dummy_func(int n)
{
    int s = 0;
    for (int i = 0; i < n; i++)
    {
        for ( int j = 0; j < n; j++)
        {
            s += i * j;
        }
    }
    return s;
}