Utility analysis - a summary of a part of The science of psychological measurement by Cohen

Critical thinking
Article: Cohen
Utility Analysis

What is a utility analysis?

Utility analysis: a family of techniques that entail a family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment.
It is not one specific technique used for one specific objective. It is an umbrella term covering various possible methods, each requiring various kinds of data to be inputted and yielding various kinds of output.

In a most general sense, a utility analysis may be undertaken for the purpose of evaluating whether the benefits of using a test outweigh the costs.

If undertaken to evaluate a test, the utility analysis will help make decisions whether:

  • one test is preferable to another test for use for a specific purpose.
  • one tool of assessment is preferable to another tool of assessment for a specific purpose
  • the addition of one or more tests that are already in use is preferable for a specific purpose.
  • no testing or assessment is preferable to any testing or assessment

If undertaken for the purpose of evaluating a training program or intervention, the utility analysis will help make decisions regarding whether:

  • one training program is preferable to another training program
  • one method of intervention is preferable to another method of intervention
  • the addition or subtraction of elements t an existing training program improves the overall training program by making it more effective and efficient
  • the addition or subtraction of elements to an existing method of intervention improves the overall intervention by making it more effective and efficient
  • no training program is preferable to a given training program
  • no intervention is preferable to a given intervention

The endpoint of a utility analysis is typically an educated decision about which of many possible courses of action is optimal.

How is a utility analysis conducted?

The specific objective of a utility analysis will dictate what sort of information will be required as well as the specific methods to be used.

Expectancy data

Some utility analyses will require little more than converting a scatterplot of test data to an expectancy table.
An expectancy table can provide an indication of the likelihood that a test-taker will score within some interval of scores on a criterion measure.

Taylor-Russell tables: increase the base rate of successful performance that is associated with a particular level of criterion-related validity.
The value assigned for the test’s validity: the computed validity coefficient.
But, the relationship must be linear.
Naylor-Shine tables: tells us the likely average increase in criterion performance as a result of using a particular test or intervention: also provides selection ratio needed to achieve a particular increase in criterion performance.

The expectancy table or chart: tells us the likelihood that individuals who score within a given range on the predictor will perform successfully on the criterion.

The Brogden-Cronbach-Gleser formula

Used to calculated the utility gain from the use of a particular selection instrument under specified conditions.
Utility gain: an estimate of the benefit of using a particular test or selection method.

Utility grain = (N)(T)(rxy)(SDy)(Zin) – (N)(C)

N =the number of participants

T = tenure

rxy = the criterion-related validity coefficient for the given predictor and criterion

Sdy = the standard deviation of performance

Zin = the mean score on the test for selected participants

C = the cost of the test for each applicant

Utility analysis: an illustration

Utility analysis has its own vocabulary.

Hit: a correct classification
Miss: an incorrect classification
Hit rate: the proportion of people that an assessment tool accurately identifies as possessing or exhibiting a particular trait, ability, behaviour, or attribute
Miss rate: the proportion of people that an assessment tool inaccurately identifies as possessing or exhibiting a particular trait, ability, behaviour, or attribute
False positive: a specific type of miss whereby an assessment tool falsely indicates that the test-taker possesses or exhibits a particular trait, ability, behaviour, or attribute
False negative: a specific type of miss whereby an assessment tool falsely indicates that the test-taker does not possess or exhibit a particular trait, ability, behaviour, or attribute

Base rate: the proportion of people in the population that possesses a particular trait, behaviour, characteristic, or attribute.

Return to investment: the ratio of benefits to costs.

Image

Access: 
Public

Image

Join WorldSupporter!
Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Institutions, jobs and organizations:
Activity abroad, study field of working area:
Countries and regions:
WorldSupporter and development goals:
This content is also used in .....

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: SanneA
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
5450