Detail. Again.

It’s only in starting to write this (and therefore filing it my perfectly constructed and maintained filing system) that I realised I’ve blogged about detail pretty recently. People will start to think that I’m obsessed.

But this time I wanted to talk about the dilemma that I come across often. I’ve just reported on a survey, and in that I had a hundred and fifty odd responses, from various grades in various locations, with both quant/rating and qual/free text questions. That’s a lot of data points. But my summary and recommendations ran to precisely 899 words. If it didn’t put quite so much spacing in it, that’d barely run to two pages.

Is that enough? Is that value for the time I’ve spent and charged on it?

I think so. (I hope so, I’ve not had feedback yet…) And it’s the approach I’ve followed for a long time, with some success. Indeed I remember a distinct stage of my career when the reports/proposals that I wrote changed from being judged on how long/comprehensive/detailed they were to how short/concise/to-the-point that they were. And a sadistic-yet-helpful boss that used to get me to summarise, strictly on one page. And then to do it again, but this time double-spaced.

So, taking this report as an example, I’ve got those many, many data points (about 2,104 since I’m doing precision today) down to two main themes. And in doing so, I can then make 10 recommendations to specifically tackle those two themes.

Now, there’s also another 50 odd pages of charts where I report on and summarise all of that data. At the appropriate time, that’s available to delve into. But if I reported on all of that, in real detail, upfront it’s going to obscure: “what’s happening?” and “what should I do?” And I think that the real value is in summarising and summarising again, until I – and by extension my client – is confident that we’ve hit the nub of the matter.

Data's just numbers, right?

I’ve written about most of the types of research that I’ve done before. In, what some people have described as, exhaustive detail. If you look that up in a dictionary, it’s a compliment. But I’m not 100% sure that they’re using it in that sense…

I’ve talked about enjoying the challenge of qualitative research. Bringing together multiple, divergent, abstract and often contradictory ideas. I like piecing that together into the story that best makes sense of it; that connects most of the ideas, most often. And within that there’s some perception, some inference, some creative license.

Data is different. Data is hard. Data is absolute. And that – tragically – excites me too. You’ve got all the pieces of the jigsaw, and they’re only ever going to fit together one way. You can’t obfuscate, you can’t flim-flam, and you can’t use your intuition. (And sometimes it’s nice to turn off and just chunk through data. Especially, say, if there’s a test match on.)

BUT, there’s still a story to be told. It’s rarely any use just taking data and pumping out some graphs. (Quite apart from anything else, your choice and layout of graphs will determine how easily people can make much sense of them. But that’s another blog….) You need to start to link themes together. What might the low-rating here mean next to the high rating there? What could cause that apparent contradiction? Why are these people scoring higher than these?

Sometimes the outcome is that you need to do further, research to understand fully. But it’ll be precise and targeted – and hence a lot cheaper than it might have been. Other times, you’ll get the answers, and context, that you needed. Data tells a story. And it’s a story I love finding.

We Don't Do Duvets

Over the past couple of years, I’ve often been asked to do desk research. And often my initial (and purely internal) reaction has been unenthusiastic. But it’s also something that I’ve done a lot of just lately, and done pretty successfully at that. So what’s the hesitation? Why do I instinctively worry?

I think it’s because there’s less control with desk research. If you’re designing a survey or focus groups, you know the outcome you want, and you can design questions to help you understand that. You might fund that your audience doesn’t have the answers, but you’ll have all of the data/reasons for why that is. You’ll get a complete picture.

With desk research, you don’t have that control. What you can find will depend on what’s been done, what’s been published and what’s available. The exact answer you want may simply not be there.

And I maybe worry that people don’t value it as much. They view desk research as “How good at Google are you?” And, of course, there’s some truth in there, but it’s more like “How persistent at Google are you?” For every source you report there are 10 others that looked good, but offered nothing. But I know where to find good demographics, salary, competitor employer information, so experience plays its part too.

My increasing comfort with desk is a realisation that it’s just the same as the other qual and quant. It’s all about telling a story. (And ideally it’s about shaping that story in advance, so that we report on what we know we can know). A story of what you were able to discover, of what isn’t available, of what the absence tells you. A story of what you can know AND what you can infer. Because that way – like with all research – you start to elevate through Data-Insight-Knowledge-Wisdom. And that’s where it has value. And that’s where I start to get enthusiastic about the work!

Procrastination

It’s insidious, isn’t it? You have the best of intentions, then it gets you – and half an hour’s gone by. Well, with no paid work today, it seems like a good day to blog about it. Because on a day like this I’ll worry that I’m not getting anything done. And the truth is that most days I worry that I’m not getting enough done, or that I’m not doing it fast or attentively enough.

Why is that? I’ve never (at time of publishing…) had the feedback “Did that really take you all day?” or “Is that it?!” Yes, I’ll get feedback and extra requests and – normally - I take that with a smile. My clients seem more than happy with what I produce and how I produce it.

So why this nagging doubt? In part, it’s because I undoubtedly do my best work when I’m up against it. There’s something about pressure or volume that puts my brain into an active, focussed state. I work quickly, creatively and I see the gaps. It’s not always comfortable, but it’s effective. On the other hand, give me a day to do a half-day job, and at about 5 o’clock I’ll realise I’ve missed the blindingly obvious. So am I creating my own pressure by wasting half the day? Or with that much space in the day is the lure of the web just too strong?

Broadly, there’s two things you can do about procrastination. You can tackle it head on: http://www.fastcompany.com/3008359/3-ways-tackle-your-procrastination-problem?partner=newsletter

Or you can use it more like a way of taking a long run-up to getting going: http://hackerspace.lifehacker.com/how-procrastination-can-become-a-productive-activity-1440113157

Me? I’m probably more of the latter. I’ve always got a list of things that I can usefully do that isn’t paid work: so that’s admin and finances, that’s new business, that’s reading article and blogs, that’s researching new tools and methods AND it’s some life admin too. It’s teaching myself touch-typing (although that in turn is getting thoroughly procrastinated right now). And at times it’s telling myself I have to complete a section, or work to a time before I check again on Surrey’s latest doomed attempt to win a four-day game.

How do you overcome the urge to waste your own time?

More of Tom Gauld’s excellent cartoons here: http://myjetpack.tumblr.com/

“That’s a stupid question”

As someone who designs questions for research, that’s not really what I want to hear. But I do. And sometimes expressed in more forceful terms than this too.

Most of the time, it’s in surveys. I can chalk it off as the reaction of someone who is too short on time to give a considered answer. But just occasionally, I’ve had it in focus groups or interviews. And then you have to ask whether the question you’ve written is as accessible as it should be? Does it make sense to the person you’re asking it of? What help do they need to break down how they feel about their employer into more manageable chunks?

In a focus group or interview, you’ve got the chance to qualify the question and help people into it. In a survey they’re on their own, so you have to be careful. So let’s look at a current question from Linked In’s Inspiration Index: http://lnkd.in/ii_nus. It’s a single question: “How often do you feel inspired by your work?” And it has some issues, that I’m sure they must have considered carefully:

• “Inspired”: is that the right word? We can see why they avoid “engaged”- that’s too hard to define. But why not “happy” or “satisfied”?
•  “Inspired”: what do they mean? Those days when you genuinely feel the love for your boss, your clients and your work? Or just the days when you “get it”?
• “Inspired”: does it matter? Do you need to be inspired in your work, any more than you need to be inspired when you’re socialising, being a parent, or just doing the dishes?
• The rating system. There’s a sliding scale from “Never feel inspired” to “Always feel inspired”, with no points in between. How does my shifting the slider “a bit” compare to the next man’s?
• The rating system. There’s no control, or comparison. It’s an absolute scale, but no definition of what, say, 73% means.
•  The timing. What’s happened that morning is likely to have an effect on my response. Will I be asked again?

But honestly, I don’t think it’s a stupid question because, what they’re looking to find out is clear enough. They’ve gone for one question to maximise response at the expense of detail, but they know a lot about the people responding anyway. In aggregate - and there’s over 100,000 responses already - the data will become fairly reliable. Potential problems with the rating system and timing will likely be ironed out in the bell-curve. And there’s standard practice to deal with the bulk response and the outliers. 

What I am saying is that even to design a one-question survey, you need to consider a lot of things. So get someone who does it a lot to help you.