Making meaning out of data…when is a number not a number and so what?
Author: A.J. (Tony) Cotton, AM
There’s so much workforce data out there that making sense of it all can be difficult, particularly when the statisticians get involved (or someone who thinks they’re a statistician). Part of the ‘problem’ comes from the way statistics is taught at university. This will seem strange to some readers but it is too 'practical'. They focus on the complexities of statistical testing at the expense of more fundamental but less technical aspects of data.
So, there are some basic things we need to know about.
First, in workforce data, numbers aren't always numbers. By this I mean that we are looking at workforce and workplace data we use numbers to organise things that aren't necessarily numbers. For example, an individual’s response to a survey question is an opinion at a point in time that we capture, categorise, manipulate and analyse as if it were a number. It is easy to forget the number’s origins and treat it as something more concrete. So, we need to be very careful about how we treat workforce data and remain mindful of it’s origins—what is actually represents.
Let me give you an example. A traditional set of possible responses to a survey question is to use a five point scale from ‘strongly agree’ to ’strongly disagree’. We allocate ‘scores’—a number from 5 to 1—to each possible response . While these might look like numbers, they’re actually not numbers in the traditional sense. If, for instance, I ‘disagree’ to a question about my satisfaction with leadership in the organisation and score a two and you respond ‘agree’ to the same question and score a four, this doesn't mean that you are twice as satisfied as I am. All this can tell us is that you are more satisfied with the leadership of the organisation than I am but it can’t tell us how much more satisfied you are than I.
Now, there is often a temptation to treat these results like numbers. We might add them up and do all sorts of other mathematical manipulations with them like calculating averages and so on. With some clever statistics you can do this but it is not for the uninitiated. And, at the end of the day, you might ask how much value this adds for purpose of making better decisions about the workforce. Indeed, as we look at finer differences in scores the meaning becomes even less clear. So, if your team has an average ‘score’ of 3.7 on a question and my team has an average score of 3.4 we can fairly confidently say that you team is more satisfied than my team but beyond that we really can’t say much at all because we’re not really looking at a number. This doesn't mean that it’s of no use rather that you have to interpret these ‘numbers’ more carefully and in context. So, a difference between two ‘numbers’ might be worth investigating but the meaning may well be different from what you expect.
As I mentioned earlier, with some really clever statistics you can say more definite things but when we’re working with more basic analysis (which is typically enough for most workforce analysis) you just need to be a little more circumspect.
One simple way to get a number that really is a number is to count things. In our example above, a simple way to get a number is to count the number of employees who agree or strongly agree with a statement and then present this as a proportion (usually a percentage) of all of the responses. In this case if 60% of your team agree with a statement but only 30% of my team agree we can say that twice as many people in your team agree with this statement. Similarly, if 60% of your team agree compared to 55% of my team we can say that your team is more supportive and that the difference is quite small.
Which leads me to my next point, how big is small? How do you know that a difference between two numbers, whether it’s between two business units or the same workgroup from one year to the next, is a real difference?
This is a critical question because if the purpose of the data is to support business decisions, which inevitably involves a resource commitment of some sort, you have to know that the difference is something that needs addressing, and not just something that reflects some random act of luck.
More to come on this.
Thanks for taking the time to read this post
Photo credit: Photo by marfis75 - Creative Commons Attribution-ShareAlike License https://www.flickr.com/photos/45409431@N00
