Article summary with A real-life, ecologically valid test of executive functioning: The executive secretarial task by Lamberts a.o. - 2010

How does one ensure ecological validity?

Assessment in neuropsychology can help to rule out or to confirm diagnoses or to decide which treatment a patient should be offered. It is also useful to predict in which way the brain injury will affect the daily life of a patient. An example of a question of this goal is: will the patient be able to resume his or her previous work and will he or she be able to resume other daily-life activities? To get the answer on these questions, tests can be used. The problem in neuropsychological assessment is that many of the tests that are available, are not valid enough to make recommendations. This is especially true for when a patient is brain-injured and there are issues with executive functions (EF). EF is about problem solving, planning, goal formation and goal management. Especially the maintenance of goals and subgoals and things such as self-initiative, self-monitoring and self-regulation are considered important when talking about EF. Ecological validity, meaning that the methods, materials and setting of the study must resemble a real-life situation as much as possible is very important in the neuropsychological context, because one of the goals is to predict the everyday functioning of patients with brain injuries. The exact definition of ecological validity for neuropsychology according to Sbordone (1996) is: “the functional and predictive relationship between the patient’s performance on a set of neuropsychological tests and the patient’s behaviour in a variety of real-world settings”.

To ensure ecological validity, one way could be to use tests that have certain items or tasks that resemble everyday life situations. This does not guarantee that the test is useful in predicting behaviour, because this has to be empirically tested. There are now several tests for testing the EF of patients, such as the Trail Making Test, the Stroop test, Wisconsin Card Sorting test, Tower of London test, and the Behavioral Assessment of Dysexecutive Syndrome.  The last one has the highest ecological validity, but is still limited in predicting everyday behaviour in patients with brain injury.

It is a fact that most tests of EF are not very good at predicting everyday behaviour. One reason for this could be that most of these tests were invented with the goal of determining whether there is or there is not a cognitive impairment. The Executive Secretarial Task (EST) aims to predict everyday executive functioning. This test is different than others, because it is based on functioning instead of on theoretical constructs. Because of this, the test ensures that the way the assessment is done (the test) is more representative for daily life. The EST is also different than other tests in that it extends over a longer period of time, that organization and prioritization of multiple tasks is tested and that participants have to deal with delayed intentions, interruptions and deadlines.

What was this study about?

In the first phase of the study it was expected that the EST would be able to discriminate between healthy people and brain-injured patients (that were already diagnosed). Another part of this study was to examine if the EST was able to determine which components of EF were most damaged. If these two expectations are met, then the EST would be very helpful to be used in neuropsychological assessment to predict a patient's’ performance in specific daily-life situations. And eventually this will lead to useful recommendations for how to overcome problems or for a certain treatment for the patient.

What conclusion could be drawn?

The EST seems to provide valuable information on the EF of brain-injured patients. It met the two expectations that it would be able to discriminate between healthy people and brain-injured patients. The EST also seemed to provide good indications of which parts of the EF were most damaged in the brain-injured patients. So, when testing the EST in a clinical population (being the brain-injured patients) the conclusion is that it is useful for neuropsychological assessment. This is true for in the diagnostic phase as for in the treatment phase, when it needs to be decided which goals the treatment has. This study has therefore demonstrated that scores on the EST are associated with executive functioning in daily life situations. There are also some negative aspects of the EST. An example is that some questions remain, such as: is it helpful in the process of selecting which job or other daily-life activities a brain-injured patient can resume? This is about the predictive validity of the EST. When this has been established, it is well recommended to use the EST in clinical practice. Other things are that it is a long test and that it is not suitable for retesting a patient: it seems to only give reliable results the first time a patient is tested. So, there are recommendations to develop an ecologically valid parallel test for EST.