Relevance Theory gives methodological tools for finding the roots of the so called content effect in Wason Selection Tasks. This study used these methods to retool classic tasks of competing content effect theories: Social Contract Theory and Pragmatic Reasoning Schemas. No significant results in the order of Relevance Theory’s predictions were found, on the contrary: facilitation effects actually led to lower performance. No significant effects in the direction of either Social Contract Theory or Pragmatic Reasoning Schemas was found either, leaving the debate between these two theories intact. Relevance Theory’s claims of designing a Wason Selection Task to be yield very high or very low performance were not substantiated in this study, although hints the method’s limitations may have been found in the process.
Primary supervisor: W.R. van Joolingen
Secondary supervisor: C. D. Hulshof
The content effect
One of psychology’s most researched topics in the past 50 years has to be the puzzling question of how different elements influence performance in the Wason Selection Task (WST). Originally created by Wason (1966), this deductive reasoning task requires subjects to solve a problem in which they have to apply a conditional rule “if P then Q” on a set of four cards containing “P”, “Q”, “not-P” and “not-Q”. According to formal logic, the only correct selection to this logical problem would be those cards that represent the falsifying instance of “P & not-Q”.
Ideally, if all humans were purely rational creatures whose brains are designed to apply formal logic in the most rational way possible, this task would lead them to select the two correct cards displaying “P” and “not-Q” representations of the problem at hand. In the case of the original task, these cards would be: E and 7 (see Figure 1). Interestingly, only an abysmally low percentage of subjects succeeds in making the correct selection in these tasks; typically less than 10-20% (Wason, 1966; Wason & Shapiro. 1971; Evans, 1982; Griggs & Cox, 1982).
Figure 1: Wason’s Selection Task (1966)
At the time Wason created his now famous task, it was already known that tasks that included more meaningful sentences and words led to improved performance when compared to abstract versions of the same task (Wilkins, 1928). This knowledge combined with the results from the WST raised the questions: how exactly does meaningful content lead to improved performance on the WST and exactly what processes make meaningful content meaningful?
The quest for answers to these questions resulted in the WST becoming more than just another task in the decades that followed. It was not so much the significance of the task’s creation, as it was the implication of its results that shook the foundations of a psychology that focused on syllogistical reasoning in that time (Wason in Evans, 1983). The low percentage of correct performance on the WST starkly contrasted the expectations of rational cognitive reasoning as it was viewed in the 1960’s.
It is no surprise then that from the 1970’s onwards, studies into the WST tried to come up with explanations for the low performance on an abstract form of the WST while familiarity with the subject appeared to greatly enhance performance. The positive effect of meaningful content on performance has been called many things. It has been dubbed the thematic facilitation effect in certain instances (Griggs & Cox, 1982; Yachanin & Tweney, 1982), but for this study its most common name will be used: the content effect.
A lot of explanations have been given for this content effect. Matching bias (Evans & Lynch, 1973; Evans, 1998) tried to explain false P & Q selections through a bias in selecting those options that were given in the task’s conditional rule. Yet the matching bias theory doesn’t hold up well when realistic materials are being used. Mental model theory (Johnson-Laird, 1983) proposes that humans do not usually use formal logic models in everyday problem solving. Formal logic can be used, but it is only one element of mental representations of the problem.
Typically, humans focus on the confirming true states rather than looking for possible falsifications. This is known as confirmation bias (Wason & Johnson-Laird, 1972) or the principle of truth (Johnson-Laird & Byrne, 2002). Whereas these and many other theories focus on a domain-general approach to answering the questions that the content effect raises, two domain-specific theories have tried to explain it in their own way. They are Social Contract Theory (Cosmides, 1989) and Pragmatic Reasoning Schemas (Cheng & Holyoak, 1985)
Pragmatic Reasoning Schemas
Pragmatic Reasoning Schemas (PRS) attempts to explain the content effect through a general set of permission and obligation schemas that govern people’s logical problem solving capability. Instead of consisting of a set of purely syntactic rules, Cheng & Holyoak (1985) describe these schemas as defined in terms of classes of goals (e.g.: taking desirable actions based on any situation) and relationships to these goals (cause and effect or prerequisites for possible actions). The basic permission schema of PRS uses deontic instances of must and may, while for the most part the permission schema fit the possible P & Q selections as formal logic would (Cheng & Holyoak, 1985). The exception is that PRS tries to block the fallacies of formal logic basic (Table 1), adds deontics to the rules and includes causal schemas as a possibility.
Table 1: Formal rules of inference and rules of the permission schema
Rule 4: If the precondition P is not satisfied, then action Q mustnot be taken
The additions of the deontic elements must and may add ease of use for everyday problem solving. For instance: Rule 1 can be rewritten as “P is true, therefore Q can happen”, allowing for more context-sensitive (MP) inferences; if you have money to pay for a product, you can buy it. It also prevents the fallacies of (AC) and (DA). For example, Rule 2 states that precondition P is irrelevant if action Q is not taken, blocking the fallacy of (DA): there is no use in denying the antecedent if the antecedent is irrelevant. Likewise, Rule 3 states that action Q may or may not be taken if precondition P is satisfied, making Q irrelevant and blocking the fallacy of (AC). Finally, failing to satisfy precondition P means that action Q must not be taken; if you don’t have money to pay for a product, you cannot buy it.
Although the rules of the permission schema, which will be the main PRS focus in this study, share similarities with formal logic, the use of deontics allows for more everyday reasoning than formal logic does. Permission rules are often more useful as heuristics than having to apply formal logic: why spend more computational resources in trying to find the validity of a statement, if you can just compare it analogously to an everyday heuristic and find the answer using much less time and mental resources?
Furthermore, causality plays a big role in everyday reasoning (for a review of the influence of causality on reasoning, see Goldvarg & Johnson-Laird, 2001). Events are often perceived as having a single cause, leading to a possible fallacy of seeing an effect as evidence for a possibly invalid cause, an (AC) fallacy. Using permission schemas, people would consider the effect as only one possible result of the cause. A cause might be the reason that an effect is noticed, it also might not be. If you get stomach aches after eating spoiled food, then spoiled food is not necessarily the only cause that created a stomach ache.
Permission schemas are both context sensitive and domain specific. This means that evoking a permission schema in a WST that has thematic content, could lead to facilitation through the use of permission rules. If a subject is familiar with the context and has experience with the specific domain, exposure to such a problem would evoke permission schemas in the subject and lead to increased performance levels, when compared to subjects where permission schemas would not be evoked. Only adding a rationale to the problem would help any subject to evoke a permission schema and perform well on a WST.
One classic example of this is the Stamp Problem (Cheng & Holyoak, 1985), where recent experience with a specific postal rule led a group of students in Hong Kong to performance levels of 90% compared to 60% levels of Michigan students who never heard of the rule. Adding a rationale to the task, giving subjects more information about why they should care about the rule, led to performance levels of 90% for both groups. This meant that context sensitivity was definitely a factor in explaining performance, through experience with the specific context. It also meant that adding thematic content in the form of a rationale facilitated problem solving for those students that did not know about the rule beforehand: contrary to the syntactic view, thematic content influenced performance while the rule remained the same. According to PRS, adding a rationale and keeping context specific prior knowledge in mind can help evoke permission schemas in subjects and therefore lead to increased performance and explaining the content effect.
Among criticisms of Pragmatic Reasoning Schemas is the claim that the used experiments contained a negation in the rationale of the task, leading to more falsifying strategies regardless of the evocation of a permission schema (Noveck & O’Brien, 1996). PRS studies as created by Cheng & Holyoak contained methodological flaws that would be addressed by Cosmides (1989) in her formulation of Social Contract Theory: subjects were allowed to flip back and forth between tasks of the Stamp problem, which could have lead to a transfer effect in the process. In absence of such transfer, no content effect would take place (Griggs & Cox, 1982).