Selecting your own questions

Does one size fit all?

We have been providing 360 degree appraisal systems for 6 or 7 years now. One trend that we have noticed is a move to having recipients have some control over the questions that are asked. At one extreme we have responded to this with a full self service appraisal solution www.myown360.com . But for most of our corporate clients the requirement is to allow the recipient to tailor their questionnaire or simply select the competencies that they should be assessed upon.

We see this as a great development. It assists the process of gaining the recipient's buy-in to their own feedback, reduces the number of questions that are not relevant, and often reduces the overall question count as there is no need for a superset of questions that everyone is asked.

A free-for-all is not the answer of course. Instead, we encourage organisations to set the overall framework - perhaps including some core competencies that are asked of everyone. Optional competencies are then selected from a range designed by HR or with our involvement.

As well as selecting competencies we are also seeing requests to select the questions that are to be asked. For a given competency the company may offer up 6 alternatives with the recipient selecting the 4 they see as most appropriate.

We beleive the benefits of an online 360 degree appraisal system are starting to be realised as the introduction of practices that improve 360 are being introduced at the same time as costs are reduced through a move from a paper-based procedure.

360 degree appraisals from Bowland Solutions

Online 360 degree appraisals

Online versus paper

As a provider of online 360 degree appraisals it will come as no surprise that we see value in moving the process away from paper-based. If you are having to make the case for an investment in a 360 system then here is a list of what we see as the key benefits. Some obvious, some less so.

• Reduced administrative overload - the key reason people approach us
• Flexibility on questionnaires - it is very simple to have subsets of questions for peers compared to direct reports, say. You could do this on paper, but I wouldn't recommend it!
• Able to monitor and chase progress easily
• Higher completion rates. I'm sure someone has worked out why, but people respond to email better than they respond to paper.
• Great flexibility on reporting - and reporting is instant
• Reduced ongoing cost
• Simple routes to get consolidated reports - that analyse strengths and weaknesses over a range of demographic information
• Integrate into your performance review process - or even your course delegate management system.
• Repeating the process is trivial

Frankly, for more than 10 people I would not consider a paper-based 360 process. And even for smaller numbers, solutions like our self-service 360 www.myown360.com make more sense than paper.

Brendan

360 degree appraisals from Bowland Solutions

Releasing the report

The big moment

Another client, another day. Today the topic was - when should the recipient get their feedback report and, more importantly, should the manager get to see it first.

For budgetary reasons and straightforward resourcing issues it was not feasible for an external or non-line manager to run the debriefing session and so the question came; should my manager see my 360 degree feedback before I did?

I boldly recommended that the manager should see it first, prepare for the debrief session, and then take the recipient through their feedback during a meeting. But our client was adamant that the recipient should be in control of the process and should not find themselves meeting their manager in the corridor and getting a "you should see what's in your 360" comment.

The ideal is, I believe, that a non-line person gives a debrief and that is the first time the person sees the report. Two key reasons; it allows the report to be understood and it allows a context to be provided. A third consideration is it allows a trained person to observe whether the recipient is emotionally affected by the feedback.

This couldn't happen here. I'm frankly uncertain. I am always nervous of someone reading their report without any assistance on understanding and context. Yet, having the recipient in control of the process is also one of our recommendations.

The client chose to have the recipient remain in control and "release" the report to their manager in advance of the feedback meeting.

Brendan

Find out more about 360 degree appraisal solutions with Bowland Solutions

How many is too many?

Deciding how many raters to have in your feedback

This is way up there in the "what do you get asked most" table. Many of our clients immediately recognise the conflict between having lots of raters - to get the most rounded feedback - and keeping rater numbers low to stop the procedure becoming too burdensome.

Let me start with our recommendation and then work back. We recommend 8-12 raters. Anything under 6 and we start to worry; anything over 15 and we worry about that as well (we worry about our clients see!)

The factors that move you toward a lower number are:

• burden (linked to how many questions there are in the appraisal - I'll write about that another time),
  
• desire to get a tight set of feedback results, and
• a desire to ensure everyone who gives feedback has something useful to say.

The factors that make you move to a larger number are:

• anonymity - another broad topic for another day, but you do need a quorum of responses if you wish to retain anonymity
• wishing to make the process as inclusive as possible
• having a range of roles you want feedback from (external peers, internal peers, customers, suppliers, etc.) - a real benefit of an online 360 degree appraisal system
  

What we would say is that big is not necessarily better. After a certain level you will find that you can't see the wood for the trees. When giving feedback you actually find the volume of information - particularly from narrative sections - overwhelming and so will the recipient.

As a sweeping generalisation, we recommend boss, self, 3-4 peers, 3-4 direct reports. We see, and recommend, a great deal of variation from that set-up but it's a good starting place.

Brendan

Find out more about 360 appraisal systems with Bowland Solutions

What the boss says in 360 degree feedback

"My boss doesn't like me"

Bowland Solutions provide an online 360 degree appraisal system . We get to see a lot of completed feedback and often we give debriefing sessions based on that feedback. What do we see?

Well, one of issues that interests academics is whether the role a person plays influences their responses. For us mere mortals, there is probably no surprise that it does.

We have found that bosses consistently score lower. Because you see the same person play out multiple roles (sometimes they respond as a peer, sometimes as boss, etc) you can isolate the role influence. Clearly you can't rely on it happening on each appraisal but, overall, bosses score lower.

In my previous article on scoring in 360 degree appraisals I warned against averaging scores from a 360. Here is another reason why it doesn't work well. If the boss is scoring in a range 1-4 and everyone else tends to score in a range 2-5 then you are not comparing apples with apples if you combine their scores.

What to do? We recommend that you use a bar chart style report that shows the boss' score clearly. When feeding back, armed with your new knowledge, you can place the boss score in context and you can see any consistent underscoring over the whole appraisal.

In a future article we will also discuss how boss' write narratives compared with others - and that gives you the main clue on how to interpret the scores.

Brendan

Find out more about 360 appraisal systems with Bowland Solutions