WMScore

Unlike widely-used Readability indicators (see [1] for a review) such as the Flesch-Kincaid Grade Level indicator ([2],[3],[4]), WMScore analyzes the structure of a sentence to assess the Working Memory 'Span Effort' required to process it. Sentences with high WMScores are undesirable when a document is intended for individuals with average- or below-average Working Memory spans.

As discussed previously, most Readability indicators indicate 'High' Readability for the well-known garden-path sentence #1. However, garden-path sentences have been studied by cognitive psychologists for decades, and are known to place a very heavy load on Working Memory (see for e.g. [5], [6] that have conducted studies of Event-related potentials (ERP) related to 'garden path' and other sentences that require 'reanalysis'). In this garden-path sentence, the reader realizes that he/ she was 'led up the garden path' only when the verb 'fell' is reached, and that the sentence has to be reprocessed from scratch.

Similarly, it can be seen that sentences 2 and 3 below have very similar, low Flesch-Kincaid Grade Level indicators (indicating ease of readability) but sentence 3 is clearly much more complex (the higher 'effort' required to parse it is reflected in the higher WMScore for sentence 3). This is largely because sentence 3 uses a complex grammatical construction called a Reduced-Form Restrictive Relative Clause (in Passive Voice) that makes it more complex to process (this was revealed in a number of tests conducted on university students since the late 1960s - see [7] and [8] for example) due to a garden-path effect. Sentence 2, on the other hand, has a simpler syntactic structure.

Notice from 1-rev and 2-rev below how a simple transformation improves the readability (and the WMScore) of these two sentences.

However, the above transformation may be easier said than done because Word Processors do not report the presence of such Reduced-form Restrictive Relative clauses in Passive Voice. This grammatical construction can be found even in modern textbooks, as authors may ignore complex sentences once they judge the Readability indicators for the passage as a whole to be reasonably low.

The WMScore scoring mechanism takes into account most of the findings from experimental studies conducted by cognitive psychologists during the past five decades (mentioned on preceding pages). It is a variant of the functional completeness/ segmentation (see [9], [10], [11], [12], [13], [14], [15]) and Locality Theory models ([16]), and is broadly based on the duration for which phrases must be kept in Working Memory before the clause is 'functionally completed'. It is computed for each sentence from a parse-tree generated by a Deep Parser. However, parser reliability degrades as sentence length increases beyond a level; this is because errors get compounded and one wrong decision leads to another. WMScore uses an approximation algorithm to handle defective parse-trees; it computes a score that may err on the higher side. Of course, one does not need an advanced algorithm to reason that extremely long sentences need to be reworked.

The WMScore needs to calibrated empirically - once this is done, we should be able to identify sentences that cannot be processed by an individual, given that individual's Working-Memory Span. As Nagel ([17]) famously argued, it is impossible for anyone to 'know' what it is like for another person to experience some phenomenon.

Contact us for more details.

References