Rating Errors

Some reasons for inaccurate assessments are:

  • confrontation avoidance
  • embarrassment about giving praise
  • not noticing good or poor performance
  • taking things for granted
  • spending too little time on assessment
  • not being familiar with or not accepting the rating scale

 

 

What about your experiences with rating behaviour?

Take a little time and reflect on your own experiences from everyday life, your workplace, or school …

Which mistakes do untrained raters often make?

 

Compare your findings with the errors on the next pages.

Note: On the following pages you will find 12 common rater errors with descriptions and explanations. Familiarize yourself with each of them.

We also recommend applying them to your own experiences.

 

Common rating errors: leniency, strictness and central tendency

  • leniency: Tendency to give ratings that are too high (inflation).
  • strictness: Tendency to give ratings that are too low (deflation).
  • central tendency: Tendency to avoid all extreme judgments and rate people as average or neutral.


 —   FINCODA Barometer rating scale  —

 

Possible reasons for leniency (inflating ratings)

Possible reasons for strictness (deflating ratings)

Possible reasons for central tendency

Keep in mind about leniency, strictness and central tendency

Leniency, strictness and central tendency assessments may not necessarily be erroneous, but could reflect the true variance of the scores.

One reason for a special variance of the scores could be selection. For instance an elite department hires only the best, or there are labour market problems (it is not possible to find excellent employees for the money the company is willing to pay).

This cannot be corrected by requiring distribution of ratings according to the normal bell curve.

Other rating errors

  • selective perception
  • primacy effect
  • recency effect
  • halo or horns effect
  • similar to me effect
  • contrast/context
  • stereotyping
  • personal biases
  • attribution

Selective perception

People selectively interpret what they see based on their interests, background, experience, and attitudes. The perceiver singles out information that supports a prior belief and filters contrary information. You have already learnt about this error during the BOT.

Recency effect

Tendency to give greater weight to recent performance and lesser weight to earlier performance.

Primacy effect

Information presented early has more impact on impressions than information presented later.

Our attention is at a maximum when making initial impressions. Once we think we have formed an accurate impression of someone, we pay less attention to later evidence. Later information is dismissed because it is not viewed as typical. Early information affects the interpretation of later information. We interpret inconsistent information in light of the first impression.

“Halo or horns” effect

The overall impression of an individual is based on a single characteristic. Someone is generally rated high or low if he is extremely high or low in one particular dimension. For example, a very attractive person gets high ratings in other dimensions.

“Similar to me” effect

Similarity leads to approval. We tend to prefer people who are similar to us. Raters are more likely to give higher ratings to people perceived as similar.

Example:

 

Attribution

Tendency to underestimate situational factors that may account for ratee’s performance and strongly emphasize a person’s personal qualities (personality or intention).

Stereotyping

Judging someone on the basis of one’s perception of the group to which that persons belongs. For example, a supervisor may stereotype by saying that young employees do not take responsibility for their work.

Example:

Contrast/Context 

Performance rating is affected by comparisons with other people. E.g. an employee may get higher ratings just because he is better than others, but he may still lag behind when judged by the standards expected. Raters often find it easier to make comparisons in the group rather than to judge the individual behaviour and judge it according to the scale. As a result, other people actually determine the final rating.

Personal bias

Discrimination based on age, sex, race, etc. Personal bias has various sources: information obtained from colleagues, beliefs and world views, social and family background, and so on.

Example:

Personal bias is often implicit. See also the Harvard Implicit research project.

Personal bias—Harvard Implicit Association Test

The Harvard IAT measures implicit attitudes and beliefs that people are either unwilling or unable to report.

The IAT asks you to pair two concepts (e.g. young and good, or elderly and good). The more closely associated the two concepts are, the easier it is to respond to them as a single unit. So, if young and good are strongly associated, it should be easier to respond faster when you are asked to give the same response (i.e. the ‘E’ or ‘I’ key) to these two. If elderly and good are not so strongly associated, it should be harder to respond fast when they are paired. This gives a measure of how strongly associated the two types of concepts are. The more associated, the more rapidly you should be able to respond.

See the following link to find out more about your implicit bias: https://implicit.harvard.edu

Summary rating errors

  • selective perception
  • primacy effect
  • recency effect
  • halo or horns effect
  • similar to me effect
  • contrast/context
  • stereotyping
  • personal biases
  • attribution
  • leniency
  • strictness
  • central tendency

Read more about rating errors on https://yourbias.is/