The Hidden Psychology Behind Feedback Scales And The Optimal One To Use

This guide is written in response to the many queries GFB receives from L&D and HR professionals who use data to manage their talent retention and development strategies. Understanding how to choose the right feedback scale is one of the most important decisions you can make when designing a 360 or employee survey.

It is impossible to recommend a single perfect response scale for every situation, as the optimum scale should meet the specific needs of the survey in question. When defining a survey scale to meet specific requirements, the key considerations are:

• Whether the scale is odd or even numbered
• Number of points on the scale
• Labelling of the points
• General reliability and validity of the scale

The Odd vs Even Scale Debate

Odd numbered scales are generally regarded as allowing for a neutral option, such as "neither agree nor disagree." Supporters of the neutral point argue that giving a "don't know" option ensures that respondents do not manufacture opinions instantaneously.

However, advocates of even numbered scales argue that in reality people are never truly neutral on issues and always have an opinion, even if they had not previously articulated it. Moser and Kalton, in their book "Survey Methods in Social Investigation," argue that there is clearly a risk in suggesting a non-committal answer to the respondent, as they believe a mid-point allows respondents to opt out, which provides uninformative data. The advantage of a scale which forces a view is that when pooled with all other responses it provides a much-needed benchmark.

The challenge with forced choice scales is that they tend to positively skew overall results. In a forced choice situation, respondents prefer to be nice, rating items positively rather than negatively. This is sometimes called the Halo Effect, where an overall feeling of like or dislike leads to uniformly high or low ratings across all features. It is possible to reduce this by alternating the direction of successive ratings, or by wording points positively so that criticism feels less confrontational.

Researchers have also raised ethical concerns about forced choice responses. Rugg and Cantril argued for the middle alternative as it provides an additional graduation of opinion, and surveys with a neutral option tend to attract higher response rates, which suggests respondents feel more comfortable with them. Ultimately, whether even or odd numbered scales are used depends on the research objective. If the goal is to clearly delineate between satisfied and dissatisfied respondents, an even numbered scale is more appropriate as it forces responses into positive or negative territory.

The Problem With Central Scores

There is a further methodological issue with the central point in a Likert-type scale in that it can be ambiguous. A central score may indicate that the respondent has no opinion, or that they are genuinely torn between competing views. This makes central scores difficult to interpret meaningfully. They could represent a large cluster of undecided respondents, or a near-even split between strongly positive and strongly negative views. Neither is the same thing, but the data looks identical.

How Many Points Is Too Many?

The number of response options affects a scale's reliability (its ability to produce consistent feedback regardless of which sample of the population is surveyed) and its discriminability (its ability to distinguish between degrees of respondent perception).

Cohen (1983) concluded that a minimum of three points is necessary, while a maximum of nine can be used effectively. Ten point scales are employed less frequently because it is difficult to make distinctions finer than a ten point scale requires, and the larger the number of choices offered, the harder it is for respondents to use them consistently. Extreme categories tend to be underused, and ten point scales are often condensed into three or five point scales for reporting anyway.

Four or five point scales are therefore generally simpler and more practical, particularly as a five point scale fits neatly with the semantic range from "very good" to "very poor." It yields a good distribution of response and allows researchers to identify differences of opinion clearly. Two and three point scales, by contrast, have limited discriminative value and are rarely recommended for satisfaction or development research.

Chang's review of previous research found differing conclusions about reliability across scale lengths, concluding that the higher the number of response options, the greater the likelihood of error, as respondents' frames of reference tend to diverge on the meaning of each point. This matters particularly in 360 degree feedback, where different individuals in different roles will have varying views of what constitutes "excellent" behaviour.

What Happens When People Cannot Rate Something?

In 360 degree and employee feedback it is often the case that respondents simply do not have sufficient information to comment on a particular behaviour. A "not able to rate" or "insufficient evidence" category must therefore be included. Without it, respondents are forced to guess, which distorts the data. Chang suggests that if respondents lack knowledge about what is being surveyed, they will over-use the end points of a longer scale, further skewing results.

What GFB’s Own Research Shows

GFB has conducted internal research into a three point scale consisting of "strength," "adequate," and "development needed," applied to a questionnaire measuring five competences with seven questions each. Respondents felt heavily constrained by only three categories and, consistent with positive response bias, were reluctant to select "development needed" on more than a handful of items. Results showed significant positive skew: 65% of responses were "adequate," 25% were "strength," and only 10% were "development needed." This clearly illustrates the power of label descriptions and confirms that a three point scale is an inadequate measuring tool in this context.

GFB's further research has shown that highly detailed descriptions of response options are more effective than general ones. For example:

This level of specificity reduces misinterpretation and allows reports to be written in concrete, pre-determined terms.

Which Labels Actually Work?

The labels attached to each point can influence the reliability and discriminability of a feedback scale significantly. Definitions can be written to offer more positive than negative options, resulting in skewed data, so care must be taken to avoid this. Using just two labels to anchor the end points can create ambiguity between them, while labelling all points avoids misinterpretation and allows respondents to understand clearly what each position means.

The Mayflower organisation, which regularly implements surveys, has identified four five-point scales found to be especially effective:

Pearson Inc. also recommend two scales for measuring requirement and expectation:

Four-point requirement scale:

Five-point expectations scale:

So What Is The Optimal Feedback Scale?

In determining which scale is best for 360 degree and employee surveys, the important issues to consider are:

• Reliability: Lissitz and Green (1975) suggest that reliability starts to level off after five points, making Likert type scales the most reliable option.

• Discriminability: Likert type scales are highly recommended as they offer enough information to distinguish between participants' differing viewpoints.

• Validity: The most valid scales are those that have been employed effectively over time. A well-established feedback scale provides more reliable data than an untested one.

• Odd vs even: Forced choice scales are most suitable in specific circumstances such as customer satisfaction research. For 360 and employee feedback, an equal number of positive and negative points alongside a neutral option tends to serve respondents better.

• Labelling: All points should be labelled to avoid confusion and minimise error caused by differing frames of reference. Concise, specific, and positively worded labels are fundamental to the success of any survey.

Although a five-point Likert type scale is generally optimum for 360 degree and employee feedback, the scale you choose must always relate directly to what is being measured and be as free from bias as possible. Only when all of these factors have been considered will you arrive at the best response scale for your specific survey.

Talk To GFB About Your Feedback Scale Design

Choosing the right feedback scale is not a minor technical decision. It shapes what your data tells you, how respondents engage with your survey, and ultimately the quality of the development conversations that follow.

If you would like expert guidance on designing your scale or survey, or if you would like a copy of GFB's full Guide to 360 Surveys, please email elise.cope@gfbgroup.com or call 0333 038 6354. You can also explore our 360 degree feedback tools and employee engagement surveys to see how GFB can support your people data strategy.