CLEAR

ERAC Q&A series: What is validity and why is it important?

Credentialing organizations are expected to provide evidence of “reliability and validity” for any tests or assessment used in the credentialing process.  Validity is considered of paramount importance—but what is it?

The concept of validity has been a topic of hot debate among psychometricians in terms of how it should be defined, conceptualized, communicated, and used.  These issues bring a sparkle to the eyes of many psychometricians, but the eyes of others tend to glaze over as the discussion ensues.  Fortunately, one does not need a psychometrician’s level of expertise in order to embrace the basic concept of validity.  Validity has to do with whether a test serves its intended purpose effectively.

Read more

ERAC Q&A series: What is reliability and why is it important?

Requests for proposals for testing services often reference the desire that the sponsoring organization will have a “reliable and valid test.”  From a measurement perspective, this request cannot be met because reliability as a concept is somewhat different from the popular perspective. Reliability refers to the level of precision or consistency of test scores, not the test itself. As an example, the reliability of scores would likely be different if calculated on two groups of examinees who have a different range of ability.
Read more.

ERAC Q&A series: What is equating and why is it used?

Even when examination forms are carefully designed to be equivalent, some variation in difficulty between forms is likely.  Variations in difficulty become especially problematic when the purpose of the examination is to determine whether someone is sufficiently competent to practice a profession or obtain a credential.  It is inherently unfair for some candidates to be given harder examinations while others are given easier examinations unless these differences are taken into account.  Equating addresses these differences by determining the equivalent passing score on multiple examination forms.  Equating is often used in conjunction with scaling but these are actually two different processes.  When used together, the equating is performed first followed by the scaling.
Read more.

ERAC Q&A series: What is scaling and why is it used?

No matter how accurately examination forms are assembled to reflect a specific degree of difficulty, there is always some variability.  The goal is to make certain that subsequent forms have the same degree of difficulty as the original form, but this is not always possible.  For that reason, a standard scale for reporting results is used.  Suppose one candidate takes a relatively easy examination and receives a raw score of 82.  During another administration, a candidate takes a relatively difficult version of the same examination and receives a raw score of 77.  Is the first candidate more competent than the second? Read more.