Chapter 5 – The Evidence for what we take in

The maximum bit rate of learning – demonstrated by Mental Athletes

The problem with measuring our rate of learning novel information, is that we are largely already familiar with almost everything we experience. It is incredibly difficult to differentiate what is new and what is already known. The exception is when one is learning information with essentially random possibilities. For example, knowing one digit in a random sequence of numbers, is of no use in predicting subsequent symbols.

The most precise evidence for a limit of physiology, is obtained from activities where subjects are pushing themselves to the limit. Top 100 metres sprinter share very similar performances. Fortunately, just as there are physical athletes, there are also mental athletes who compete with each other at international events. The majority of these events involve rapid memorising of symbols or images (followed by their subsequent accurate retrieval from memory).

There are two aspects to memorising; 1/. taking in the information (learning), and 2/. consigning it to memory. In events where very long sequences of symbols must be memorised, the task is harder and dominated by the second aspect, so the bit rates observed are lower, and do not represent a limited learning rate. However, for short duration events, especially simple mental arithmetic tasks, we see similar rates for different people and tasks, suggesting this is a measure of our maximum rate of learning.

How we characterise their various performances in bits per second

The bit rate of physical skills:

(See Chapter 6 – The evidence from what we do)

More evidence comes from experiments to determine the maximum information rate of subjects performing repetitive skills requiring precision. When the precision is expressed as a number of bits, the information rate in bits per second appears to be remarkably constant over a vast range of experimental conditions.  This suggests that the speed of the task is limited by the ability to observe that each stage has been completed, a visual learning task.

Similar results have been obtained from skills involving head and foot movements

The bit rate of language:

The third source of evidence comes from the bit rate of language taking the redundancy into account, as determined by Claude Shannon.

He estimated the average number of bits per symbol needed to encode language, through experiments with human predictors. This gave an information rate between 0.6 and 1.3 bits per character in English. Later work refined this figure to 1.1 bits per character. This is far less than the 4.7 bits required to define a single character out of an alphabet of 26 characters plus a space.^[i] So here is an explanation of Predictive Text half a century before it became commonplace on our mobile phones.

Shannon’s figures for bits per character could be even lower if the total context of a message were taken into account. You can probably think of times when you struggled to understand many words in an overheard conversation, until you recognised the context, and then suddenly it made sense. Equally when we are trying to read some almost unintelligible handwriting, recognising one word can lead to an unravelling of the whole sentence.

When we use an eye tracker to monitor our point of gaze while reading, we can observe that our point of gaze jumps from word to word along a line of text. We skip over most of the simple words like “the” and “of”, as the text can be understood without them; they can be inferred. But every so often our point of gaze jumps back along the line, back several words enabling us to re-read a difficult word. It is likely that our mind has suddenly realized that it has failed to make sense of the sentence and goes back to correct a misread word (mostly subconsciously). So we can read faster if we allow ourselves to make a few errors that can be picked up later and corrected. The task of reading random words is very significantly slower than with regular prose as we do not know what to expect.

We can use this 1.1 bit per character figure to calculate the speed of human communication as an information rate in bits per second. This is typically characterised in Words Per Minute (WPM), with an assumed average of five characters per word.^[ii] So for example, a speed of 120 WPM would be 120 x 5/60 = 10 characters per second, and using the above estimate of 1.1 bit per character, this would be 11 bits per second.

Although the first examples above were based on the characters of the written alphabet, the latter example comprised of English words could equally well be written or spoken. Similar information rates have been found to apply to verbal language. So what information rate can we absorb through listening to speech? Audio-books generally “speak” at around 150 WPM, which corresponds to about nearly 14 bits per second. People can read text aloud at twice this rate, but it is very difficult to determine the degree of redundancy and the real error rate when reading material in which much is familiar to the reader. Pierce & Karlin of Bell Labs concluded that 43 bits per second can be transmitted under certain conditions and that the speed of word recognition appears to be a more severe limiting factor than the physiology of articulation.^[iii]

43 bits per second for language is somewhat faster than those measured for the memory and physical skill tasks described above. This might suggest that our brain is more highly optimised for language, as it has been such a key factor in the development of modern humans. However, it is likely that the estimates of language redundancy used in the calculation may be too low. Of course the redundancy we describe only applies when we are familiar with a language. When we first encounter a language that is foreign to us, the communication rate becomes very low, and if we are multilingual, we must spot which language is being used before we can apply our knowledge of redundancy.

^[i] The number of bits per character is the binary logarithm of the number of possibilities (= log₂N), where N is the number of different symbols in the set, in this case 26

^[ii] “Analysis of Text Entry Performance Metrics”, Ahmed Sabbir Arif, Wolfgang Stuerzlinger, Dept. of Computer Science & Engineering, York University, Toronto, Canada.

^[iii] “Reading rates and the information rate of a human channel”, Pierce, J. R., & Karlin, J. E. (1957), Bell Systems Technical Journal, 36, 497-516.