Getting them to do it

How to get a 95% and above completion rate

A number of our clients approach us because they have had a poor experience with 360 feedback and yet they still see it as a tool that will offer effective feedback for their team or organisation. One of the key issues they face is that they cannot get enough people to complete the feedback.

Here are some of our hints and tips that we provide people on ensuring they hit a high completion rate.

Let me get the "you would say that, wouldn't you" one out of the way. We recommend you put the 360 online. Bowland Solutions provides online 360 degree feedback so there is no surprise there. Now, lets get a list of reasons why this will help and what you need to do to increase that feedback rate.

• Make sure the questions, the branding, the reports, the communications are all tailored - don't use some off-the-shelf set of questions and standard set of reports. People spot quickly that this is not quite right for them.
  
• Keep the questions limited to just what you need to get the feedback required. As a rule of thumb, something like 30 questions is often right.
• Keep the numbers giving feedback to a sensible amount
• Watch out for the peer group - they often have the most feedback to give.
• Limit to 3 peers per recipient
• Consider asking the peers a subset of the questions
• Communicate why this is a good thing
• Give people enough time to complete - we recommend 3 weeks
• If you can - make the recipient the person in control of the process. That way they will chase down their respondents for you.
  
• Chase them by email
• Chase them by email again
• Phone them

We hit 95% completion feedback rates - not every time, but often enough to know that it is readily achievable.

Brendan

Find out more about 360 degree appraisal solutions with Bowland Solutions

Technorati Profile

How many is too many?

Deciding how many raters to have in your feedback

This is way up there in the "what do you get asked most" table. Many of our clients immediately recognise the conflict between having lots of raters - to get the most rounded feedback - and keeping rater numbers low to stop the procedure becoming too burdensome.

Let me start with our recommendation and then work back. We recommend 8-12 raters. Anything under 6 and we start to worry; anything over 15 and we worry about that as well (we worry about our clients see!)

The factors that move you toward a lower number are:

• burden (linked to how many questions there are in the appraisal - I'll write about that another time),
  
• desire to get a tight set of feedback results, and
• a desire to ensure everyone who gives feedback has something useful to say.

The factors that make you move to a larger number are:

• anonymity - another broad topic for another day, but you do need a quorum of responses if you wish to retain anonymity
• wishing to make the process as inclusive as possible
• having a range of roles you want feedback from (external peers, internal peers, customers, suppliers, etc.) - a real benefit of an online 360 degree appraisal system
  

What we would say is that big is not necessarily better. After a certain level you will find that you can't see the wood for the trees. When giving feedback you actually find the volume of information - particularly from narrative sections - overwhelming and so will the recipient.

As a sweeping generalisation, we recommend boss, self, 3-4 peers, 3-4 direct reports. We see, and recommend, a great deal of variation from that set-up but it's a good starting place.

Brendan

Find out more about 360 appraisal systems with Bowland Solutions

Ratings versus narrative in 360 degree appraisals

When most people think of 360 degree feedback, they think of rating people against a set of statements. Often these scores are then averaged (but see my previous blog on the problems of averaging in 360 feedback) and presented back as a wholesome view of the individual.

When completing my MBA, I wrote a thesis on the narrative element of 360 degree feedback. If any school friends are reading this then, yes, I do still enjoy sport, a beer, and other normal activities! In writing that document I came in with some bias that from my experience the narrative parts of a 360 often contained richer information than the rating scales. The thesis only confirmed this, with the following key learning points.

• The narrative is often used to place the ratings in context, e.g. "Although I answered 'poor', this relates to a specific incident where Jenny...."
• The narrative can be helpful in drawing out the way each role or person is scoring (see What the boss says in 360 degree feedback)
• Without the narrative it is very difficult to give feedback to the individual....you know how people feel about the person but you do not know why nor the impact of the identified behaviour.

Although the narrative element suffers again from role bias (bosses use directional language such as "must", "needs to" whereas direct reports soften any blows "sometimes", "Jenny is brilliant but could perhaps...") it is much easier to interpret when giving feedback.

We recommend that you increase the narrative elements of your 360, preferably having one narrative question per competency. That question should be written in a way that asks for additional commentary, some examples and the impact of the behaviour.

Brendan

Find out more about 360 appraisal systems with Bowland Solutions

What the boss says in 360 degree feedback

"My boss doesn't like me"

Bowland Solutions provide an online 360 degree appraisal system . We get to see a lot of completed feedback and often we give debriefing sessions based on that feedback. What do we see?

Well, one of issues that interests academics is whether the role a person plays influences their responses. For us mere mortals, there is probably no surprise that it does.

We have found that bosses consistently score lower. Because you see the same person play out multiple roles (sometimes they respond as a peer, sometimes as boss, etc) you can isolate the role influence. Clearly you can't rely on it happening on each appraisal but, overall, bosses score lower.

In my previous article on scoring in 360 degree appraisals I warned against averaging scores from a 360. Here is another reason why it doesn't work well. If the boss is scoring in a range 1-4 and everyone else tends to score in a range 2-5 then you are not comparing apples with apples if you combine their scores.

What to do? We recommend that you use a bar chart style report that shows the boss' score clearly. When feeding back, armed with your new knowledge, you can place the boss score in context and you can see any consistent underscoring over the whole appraisal.

In a future article we will also discuss how boss' write narratives compared with others - and that gives you the main clue on how to interpret the scores.

Brendan

Find out more about 360 appraisal systems with Bowland Solutions

360 degree appraisals and the Russian judge

It’s a 5.6 from the Russian judge

The benefit of averaging in 360s has been likened to the removal of rogue scores from ice dancing competitions. Take 9 scores, remove the two extremes and average the rest and noone can deny Torville and Dean the gold. But is this valid for a 360? Aren’t we interested in why two people scored at the extreme? Wouldn’t we be better reflecting back the detail of the feedback rather than averaging out the variety?

It seems that people like numbers - it gives clarity. Bosses like to feel it is getting to the nitty gritty of someone's performance. I'm not so sure.

First - you lose something. If three people say I am poor and three people say I am fantastic then being told that overall I am average simply isn't correct and, worse, is misleading.

Second - the average isn't correct. It assumes that people evenly score on the range. But, particularly us Brits, people don't like scoring at the extremes....so, when they do, it needs to carry more weight.

Third - 360 degree feedback is meant to give you a rounded opinion, a range of feedback. Leave the detail in, is what I say.

And finally, the conversations from the 360 are often painful. They start with "I'm 3.7 on Leadership and 3.6 on Customer Service, so I need to work on Customer Service". The precision of numbers is misleading.

Together with colleagues, I set up a company concentrating on 360 degree appraisal to give companies a chance to handle 360s effectively and efficiently. Our recommendation is always that the feedback report should give a simple representation of what has been said and keep numbers away. If there is a craven need for a score - then use something like a spidergraph to show an overall picture but don't create scores for each behaviour.

Brendan

Find out more about online 360 degree appraisal systems with Bowland Solutions