Digitisation, OCR and retrieval are separate steps
Three processes are often treated as one, and the confusion hides where the problem actually sits.
Digitisation produces an image of the original, often from microfilm rather than the paper itself in the case of historical newspapers. The quality of that image depends on the source condition, the scanning resolution, the file format and image preparation such as contrast adjustment and de-skewing.
OCR turns the image into text. It involves layout analysis, which finds columns, headlines and text blocks; segmentation of lines, words and characters; and recognition of each character, usually helped by language models or dictionaries. Many large digitisation programmes store the output in ALTO XML, a standard maintained under the auspices of the Library of Congress. ALTO records each recognised word with its position on the page and, optionally, a confidence value.
Retrieval is what the search system does with that text. It indexes it, matches queries against it and ranks the results. Retrieval can be tuned to tolerate some errors, but it cannot recover words that OCR has turned into unrecognisable strings.
A collection can therefore be fully digitised, in the sense that every page image is available, and still be partly unsearchable.
Character accuracy, word accuracy and why the difference is large
OCR quality is usually expressed in one of two ways.
Character error rate (CER) is the proportion of characters that would need to be inserted, deleted or substituted to turn the OCR output into the correct text. Character accuracy is its complement.
Word error rate (WER) applies the same idea to whole words. A word counts as wrong if any of its characters is wrong.
For searching, word-level accuracy is what counts, because users search for words. The two measures can diverge sharply. If errors were spread evenly, a 95% character accuracy applied to an average word of six characters would leave roughly three quarters of words intact. In practice errors cluster, which can make the gap smaller or larger, but the principle holds: a respectable-sounding character accuracy can hide a much weaker word accuracy.
A study of the British Library's 19th Century Newspapers collection, published in D-Lib Magazine in 2009 by Simon Tanner, Trevor Muñoz and Pich Hemy Ros, shows how wide the gap can be. The collection then held about two million pages from 48 national and regional newspapers. The team measured OCR quality on a sample of about 1% of the pages.
html
<table>
<thead>
<tr>
<th>Measure (19th Century British Library Newspapers sample)</th>
<th>Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td>Character accuracy</td>
<td>83.6%</td>
</tr>
<tr>
<td>Word accuracy</td>
<td>78%</td>
</tr>
<tr>
<td>Significant word accuracy (excluding very common words)</td>
<td>68.4%</td>
</tr>
<tr>
<td>Number groups</td>
<td>64.1%</td>
</tr>
<tr>
<td>Words beginning with a capital letter (mostly proper nouns)</td>
<td>63.4%</td>
</tr>
</tbody>
</table>The pattern is the important finding. The more useful a word is for searching, the less likely OCR is to have read it correctly. Common function words are short and frequent, and language models help recognise them. Names, places and numbers, the things researchers most often look for, are less predictable and therefore more often wrong. Only about 63% of capitalised words, largely names, were correct.
The authors suggested that above roughly 80% word accuracy, a search engine using fuzzy matching can compensate for much of the error, helped by the fact that important words often appear several times in an article. Below that level, they argued, search performance falls away steeply. On their figures, the collection's significant word accuracy was well below that point.
Confidence scores are not accuracy
Many digitisation programmes report OCR quality using the confidence scores produced by the OCR software. These scores are not the same as accuracy.
Rose Holley, then manager of the Australian Newspapers Digitisation Program at the National Library of Australia, made this point clearly in a 2009 D-Lib Magazine article. She described a sample of 45 pages from the digitised collection, spanning 1803 to 1954, in which raw OCR character confidence ranged from 71% to 98.02%. She also stressed that the software does not know whether a character is correct. It only estimates how sure it is. The only way to measure true accuracy is to compare OCR output with a verified transcription, which is time-consuming.
The distinction matters when reading any quality figure for a digitised collection. A reported accuracy may be measured accuracy against ground truth, it may be OCR confidence, or it may be an estimate derived from a small verified sample. These are different measurements and cannot be compared directly.
Holley's article also listed the factors that drive OCR quality in historical newspapers: the condition of the source and the microfilm, scanning resolution and file format, image processing, page layout complexity, fonts such as gothic or Fraktur types, and age. Older newspapers generally produced lower accuracy than newer ones, and newspapers generally scored lower than books.
What OCR errors do to real searches
The most direct evidence of retrieval impact comes from search logs. In a paper presented at the 2017 ACM/IEEE Joint Conference on Digital Libraries, Guillaume Chiron, Antoine Doucet, Mickaël Coustaty, Muriel Visani and Jean-Philippe Moreux studied Gallica, the digital library of the Bibliothèque nationale de France. They collected about 28 million user queries over four months, after setting aside a much larger volume of automated robot traffic. They compared the terms people searched for with a corpus of 12 million characters of OCR text aligned with a verified transcription.
Their main finding was that about 7% of the terms users searched for were likely to miss documents because of OCR errors. Those terms included around 2,000 common search terms, each searched at least 35 times in the period. In their sample, about 2.5% of tokens were affected by alphanumeric OCR errors, with average error rates of roughly 1% to 4% depending on the source. They also noted that error rates of up to 10% of characters had been reported for some historical newspaper documents.
The figure of 7% should be read carefully. It does not mean that 7% of searches failed. It means that, for about 7% of the terms people searched for, some relevant documents were probably invisible because the term had been misrecognised in them. For a researcher looking for every mention of a particular person, that kind of systematic gap matters a great deal. For a casual user wanting a handful of examples, it may go unnoticed.
Why ranking can survive when completeness does not
An older line of research reached a finding that looks reassuring at first. In a 1996 study in Information Processing and Management, Kazem Taghva, Julie Borsack and Allen Condit compared retrieval on OCR text with retrieval on corrected versions of the same documents. Average precision and recall were not measurably affected. The ranking of relevant documents did change under some weighting schemes, and relevance feedback could not compensate for badly degraded documents.
The reason average performance held up is redundancy. A long document about a topic usually mentions its key terms several times. If OCR corrupts one occurrence but not the others, the document still matches. Tanner's team made the same point about repeated significant words in newspaper articles.
That redundancy does not protect every kind of search. Three situations are particularly exposed.
Short texts. Advertisements, notices, letters and brief news items may contain a name or keyword only once. If that occurrence is misread, the item cannot be found.
Rare names and terms. A person who appears in a single column of a single issue, perhaps in a court report or shipping list, has no second chance.
Exhaustive research. Historians, genealogists and computational researchers often need every occurrence, not just the best matches. Average precision says little about how complete the result set is.
Averages hide this. A system can score well on standard retrieval measures while consistently failing the queries that matter most to specialist users.
Beyond keyword search
OCR quality affects more than simple keyword matching. Digital humanities research increasingly applies computational methods to digitised text, and those methods inherit its errors.
A 2020 study by Daniel van Strien and colleagues, presented at the International Conference on Agents and Artificial Intelligence, measured the effect of OCR quality on six downstream tasks: sentence segmentation, named entity recognition, dependency parsing, information retrieval, topic modelling and fine-tuning of neural language models. OCR errors had a consistent effect across the tasks, and some were harmed more severely than others.
Named entities are a particular concern because names are what many users search for. A 2022 article in Natural Language Engineering by Ahmed Hamdi and colleagues examined how different levels and types of OCR noise affect named entity recognition and linking. It showed that OCR errors degrade both, and it offered recommendations on the document quality and post-correction needed for reliable results.
Not all evidence is discouraging. Research by Giovanni Colavizza for the National Library of the Netherlands, published under the title "Is your OCR good enough? Probably so", found that for some Dutch collections and some tasks, such as topic modelling and document classification, moderate OCR noise had a limited effect. The right conclusion is that the impact depends on the task. Aggregate analyses of large collections can tolerate noise that would defeat a search for one rare name.
Ways to reduce the damage
Libraries and researchers have several options, each with its own costs and limits.
html
<table>
<thead>
<tr>
<th>Approach</th>
<th>What it does</th>
<th>Strengths</th>
<th>Limitations</th>
</tr>
</thead>
<tbody>
<tr>
<td>Re-running OCR with better engines</td>
<td>Processes the existing images again with improved recognition and layout analysis</td>
<td>Can raise quality across a whole collection without new scanning</td>
<td>Limited by the original image quality; costly at scale; old and new text versions must be managed</td>
</tr>
<tr>
<td>Fuzzy and approximate matching</td>
<td>Matches query terms to similar strings in the index</td>
<td>No change to stored text; recovers many near-misses</td>
<td>More false matches; weak for short words and heavily garbled text</td>
</tr>
<tr>
<td>Volunteer text correction</td>
<td>Members of the public correct OCR text line by line</td>
<td>Very high accuracy on corrected text; builds public engagement</td>
<td>Coverage is uneven and follows volunteers' interests</td>
</tr>
<tr>
<td>Automatic post-OCR correction</td>
<td>Software, increasingly language models, corrects OCR output</td>
<td>Scales to large collections; recent methods substantially reduce error rates in experiments</td>
<td>Risk of plausible but wrong corrections; benchmark results may not transfer to other collections</td>
</tr>
<tr>
<td>Exposing quality information to users</td>
<td>Shows confidence or quality indicators alongside results</td>
<td>Helps users judge how complete their results are likely to be</td>
<td>Confidence is not accuracy; many interfaces do not show it</td>
</tr>
</tbody>
</table>Human correction at scale
The best-known example of volunteer correction is Trove, the National Library of Australia's discovery service. When its newspaper text correction feature launched in a soft release in July 2008, roughly 1,200 people corrected about 700,000 lines of text across 50,000 articles in the first twelve weeks, according to Holley's account. The scheme has continued since and is often cited as one of the most successful crowdsourcing projects in the cultural heritage sector.
Volunteer correction follows its volunteers. Family historians correct the notices and districts that matter to their research, so corrected text concentrates in some titles and periods while large parts of the collection remain uncorrected. It improves particular paths through a collection rather than the collection as a whole.
Language models for post-OCR correction
The most active research area is automatic correction using large language models. A 2024 paper by Alan Thomas, Robert Gaizauskas and Haiping Lu, presented at a workshop on language technologies for historical and ancient languages at the LREC-COLING conference, tested this on a parallel corpus of 19th-century British newspaper text known as BLN600. An instruction-tuned version of Meta's Llama 2 reduced the character error rate by 54.51%, compared with 23.30% for a fine-tuned BART sequence-to-sequence model.
Results like these are encouraging, but they need careful interpretation.
A benchmark is not a collection. BLN600 is a specific corpus with specific error types. Performance on another language, period, typeface or kind of document may be very different.
Relative reduction depends on the starting point. Halving the error rate means something different for text that starts with 5% character errors than for text that starts with 30%.
Generative models can invent plausible text. A correction model may turn a garbled surname into a common surname, or a misread place name into a better-known place. The result looks correct and reads naturally, but it is wrong, and a reader cannot tell from the corrected text alone. For an archive whose value depends on fidelity to the source, that risk is serious.
Research results are not production deployments. Applying language-model correction to millions of pages raises questions of cost, versioning, provenance and quality control that experiments do not answer.
The cautious approach emerging in the field is to treat machine-corrected text as an additional layer rather than a replacement. The original OCR is kept, the corrected text is labelled as such, and both remain linked to the page image, which remains the authoritative witness.
What users of digitised archives can do
For researchers who depend on digitised collections, a few practical habits reduce the risk of being misled by OCR.
Treat a zero result as uncertain. A search that returns nothing shows that the term was not found in the OCR text. It does not show that the term never appeared in the source.
Search variant spellings and likely misreadings. Historical OCR commonly confuses similar letters and letter combinations, such as "rn" and "m", "c" and "e", or "li" and "h". The long s used in older printing is often read as "f". Searching for common misreadings of a key name can recover missed documents.
Use wildcards and fuzzy search where the interface supports them, and be aware that they bring more irrelevant results.
Check the OCR text, not only the image. Many interfaces let users view the underlying text. A few minutes spent reading it shows how reliable searches in that title and period are likely to be.
Report the method in research. Studies that count mentions or trace word usage over time should state that they used OCR text and acknowledge that error rates vary between titles and decades. A change in frequency can reflect a change in OCR quality rather than a change in history.
Digitised is not the same as searchable
Mass digitisation has made historical sources accessible at a scale that would have been unimaginable a generation ago. The Library of Congress's Chronicling America collection passed 15 million pages in 2019, and national libraries around the world hold comparable or larger newspaper collections. That access is real. But the text layer between the page image and the search box is an interpretation, made by software working on difficult material, and its errors are not random. They fall most heavily on names, numbers and rare words, which are exactly what many users want to find.
Better OCR engines, fuzzy retrieval, volunteer correction and machine correction can all narrow the gap. None of them closes it completely. Understanding where the gap lies, and treating a search result as a sample rather than a census, remains part of using a digitised archive well.
