ViewWiki

The world, a little closer

Algorithms and sorting.

Sorting eight blocks by hand is easy. Sorting a million takes a method. Why do some methods finish in a blink while others take hours?

Explore the scene Interactive article

01 / The experiment

Same blocks. Different method.

Pick an algorithm, grow the list and change the starting order, then choose Growth to see how the comparisons pile up.

Still of the Algorithms and sorting 3D scene, which explores: Why does sorting twice as many blocks take bubble sort four times as long?
Interactive scene

Loading the interactive version…

The explanation below is available while the scene loads.

Two raised blocks are being compared; a blue bracket means they are in order, an orange one that they swap. Each list size plays at its own steady pace.The counts are the exact operations of four textbook programs; real libraries add refinements, and time also depends on memory, caches and the data.

02 / Three methods

Every pair, or halves.

Comparisons needed to sort 8, 64 and 1,000 blocks. The bars share a scale that grows tenfold per step; the dashed mark is n·log₂n.

Every pair
28n = 82,016n = 64500kn = 1,000

Bubble & insertion

Compare neighbours and swap, or slide each block into place. On a reversed row every pair gets compared: double the blocks, four times the work.

8 blocks
28, worst case
64 blocks
2,016, worst case
1,000 blocks
499,500, worst case
Best case
n − 1, already sorted
Worst case
n(n − 1)/2, reversed
Extra memory
None: sorts in place
Halve and merge
17n = 8321n = 648,977n = 1,000

Merge sort

Split the row in half until single blocks remain, then merge sorted halves. About log₂n rounds of merging, each comparing every block at most once.

8 blocks
17, at most
64 blocks
321, at most
1,000 blocks
8,977, at most
Best case
about (n/2)·log₂n
Worst case
about n·log₂n
Extra memory
A second row as long as the first
Split by a pivot
17n = 8361n = 6411kn = 1,000

Quicksort

Put shorter blocks left of a pivot and taller ones right, then sort each side. Fast on shuffled rows; with the last block as pivot, a sorted row ruins every split.

8 blocks
17, average
64 blocks
361, average
1,000 blocks
10,986, average
Best case
about n·log₂n, even splits
Worst case
n(n − 1)/2, sorted or reversed
Extra memory
A stack of ranges, typically about log₂n deep
The big idea

Comparing every pair grows with the square of the list; splitting it in halves grows only a little faster than the list itself. At a thousand blocks that is half a million comparisons against about ten thousand.

n²/2 vs n log n

A row of blocks and one question

Put a row of blocks in order of height. With eight blocks you just look, but a computer cannot see the whole row at once. All it can do is pick up two blocks, ask which is taller, and move them. A method for doing that is an algorithm: a list of steps precise enough for a machine to follow without thinking.

Every sorting algorithm in the Scene is built from the same move. When two blocks rise under a bracket they are being compared. A blue bracket means they are already in order; an orange one means they swap. The brass plaque counts every comparison, and since comparing is the work every method has to do, that count is the fairest way to say how much work a method did.

Four ways to sort

Bubble sort walks along the row comparing neighbours and swapping any pair that is out of order. After one pass the tallest block has bubbled to the end; after the next, the second tallest; and so on. If a pass makes no swaps the row is sorted and it stops.

Insertion sort works like sorting a hand of cards. It takes the blocks one at a time and slides each one left past every taller block until it meets a shorter one. The left part of the row is always sorted.

Merge sort splits the row in half, sorts each half, and merges the two sorted halves by repeatedly taking the shorter of their two front blocks. The halves are sorted the same way, down to single blocks, which are sorted already. In the Scene each merge builds its result on the lower shelf, then lifts it back.

Quicksort picks one block as the pivot, here the last block of the stretch it is sorting, marked with a small orange pin. It moves every shorter block to the left of the pivot and every taller one to the right, which puts the pivot in its final place. Then it sorts the two sides the same way.

Try it: choose Bubble and slide List size up to 64 blocks, then switch the Algorithm to Merge. The same 64 blocks take bubble sort about 2,000 comparisons and merge sort about 300.

Back to the experiment

Counting the comparisons

Bubble sort on a reversed row has to compare every block with every other one. With n blocks there are n(n − 1)/2 pairs: 28 for 8 blocks, 2,016 for 64. Double the row and the work roughly quadruples. Insertion sort does about half that on a shuffled row, but it grows the same way. Computer scientists write this growth as O(n²), read “order n squared”.

Merge sort does less work because every comparison in a merge settles one block’s place in the merged run for good. The row can only be halved about log₂n times (6 times for 64 blocks), and each round of merging compares each block at most once, so the total is at most about n × log₂n: 321 comparisons at most for 64 blocks. Quicksort on a shuffled row averages a little more, about 360 comparisons for 64 blocks. This is O(n log n) growth, and no method that works only by comparing pairs can beat it, even on average.

Choose Growth under Show how it grows to see the two curves side by side. The orange ribbon is n(n − 1)/2, the blue one n log₂n. The white curve is the chosen algorithm’s count at every list size, averaged over 24 starting rows of the chosen kind, and the dot climbs as this run counts.

The starting row matters

The same algorithm can be fast or slow depending on the row it is given. On a Nearly sorted row, insertion sort barely moves anything: each block is compared once or twice and stays put, so the count is close to n. That is why insertion sort is still used for small or almost-ordered lists.

The quicksort here is the opposite. Choosing the last block as the pivot works well on a shuffled row, but on a sorted or Reversed row the pivot is always the tallest or shortest block left. Every split then puts nothing on one side, and the count climbs to n(n − 1)/2, as bad as bubble sort. Real libraries pick the pivot from the middle, from a sample, or at random, and some switch to another method when the splits go wrong.

What this model leaves out

The counts are exact for these four programs, and each of them could be written a little differently: a bubble sort without the early stop, or a merge sort that skips merging halves already in order, would count differently. Comparisons are not the whole cost. Merge sort needs a second shelf as long as the row, while the others sort in place; swapping or copying takes time too, and on a real computer the order in which memory is touched can matter as much as the count. Real data also brings repeated values and records that must keep their original order, which these distinct blocks avoid. Finally, the Scene plays each list size at its own pace, faster for longer lists, so compare times on screen only at one list size.

Reuse with credit to ViewWiki and a link to this page.

04 / Learn this in a course

Practise it, step by step.

These 3 lessons teach the topic with checked questions, in course order.

  1. The steps of an algorithm Computer science · Algorithms and efficiency
  2. Comparing sorts Computer science · Algorithms and efficiency
  3. Growth rates and big-O Computer science · Algorithms and efficiency