Archive for the ‘ Cautionary tales ’ Category

Another sanctimonious rave

The national reps for ESOMAR recently were asked to outline some of the challenges facing market research in our part of the world. Fortunately, there was a word limit so here are mine (both of which slosh around in the Threats and Weaknesses SWOT buckets I’m afraid).


1… Inertia in adjusting to the higher expectations of clients who are still clamouring for creative solutions to their marketing problems rather than the typical output from “command and control” type surveys. We all claim to be delivering “insight” (the most hackneyed word in the researcher’s lexicon) but most don’t do this regularly enough. The vast majority of researchers don’t make it into the boardroom because our work doesn’t warrant it. No wonder clients are going outside ESOMAR and MRS membership lists for their answers and ideas. 2… A decline in standards in two key areas: (a)  Sampling. This is not a criticism of online panels – at least we understand and accept the limitations here. There has been a gradual decline in sampling standards offline as well. The vast majority of surveys nowadays cannot have margin of error calculations done because they are not representative; most are convenience samples and response rates are too low. It’s simply a quality issue – garbage in, garbage out. (b) Reporting. PowerPoint is the villain in research as it is in the military where it is referred to as “hypnotising chickens”. It gives the illusion of understanding. As Gen. James N. Mattis of the US Marine Corps said recently “PowerPoint makes us stupid”. Most researchers fall back on it because it’s the quickest and easiest way to present. It encourages superficial analysis as it obviates the need for a clear narrative; clarity and narration require analysis, synthesis and imagination. But the chickens are waking up. Love these new infographics and data visualisation techniques. For a convenient update on these catch the next #NewMR  webinar  on 28 June 2013 to sharpen up your presentations. Go to banner_mail_600-1And a round of applause Ray Poynter and Sue York of The Future Place who do this for virtually nothing except the warm glow of helping the research community.


Ten Rules for Young Researchers

Back in the analogue days of the Seventies we had to sell the idea of market research to suspicious clients from leaky offices across SE Asia that had non-existent phone systems, intermittent powers supplies and all the other watch-them-glaze-over-old-Asia-hand diatribes… the golden era when the research business wasn’t really a business. But you’ll be pleased to see that I’ve managed to distill this sanctimonious rave into Ten Shortish Rules…

Rule 1: Maintain your incredulity. Don’t believe anything that’s recorded in a questionnaire, said in a focus group or reported in a survey unless you have personally validated it. Several times.

Rule 2: Ask sensible questions. Respondents aren’t idiots. And you’ll always get responses to a poorly constructed question; their answers just won’t mean anything.

Rule 3: Data aren’t the end result… reliable data are important but even “good” information isn’t important per se. Information is the manure – the fertiliser for ideas… step in it, walk around it but use it to understand what’s happening and what the client should do.

Rule 4: Make definite recommendations. Go beyond the data. Remember, knowledge is adjudicated information and you have the authority to adjudicate. Some clients may not want your opinions but give them judiciously. All of us have numerous examples of studies where clients have ignored our advice and lost lots of money.

Even an Old Economy researcher can tell when a New Economy idea bombs in research. We once watched a dotcom client drop $10 million in three months because they disagreed with the fairly obvious prognosis that Singapore didn’t need their version of an online department store.

The late Dr Peter Kenny (the brilliant and wonderfully eccentric Aussie researcher whose letterhead featured a naked picture of himself – as one does – on the basis that it had impact and opened doors) once told me to “push clients to do what you suggest…they need the confidence to act in a definite way. Get them to do anything, anything at all and they’ll get a result!”

Rule 5: Be brave with your knowledge. Don’t be afraid to deliver bad news. Only the ancient Romans cut out the tongues of the bearers’ of bad news. If in doubt, avoid Italian clients.

Predictions are difficult. Certainly don’t expect consumers to predict the future for you – they can’t.

Rule 6: Write properly. Research reports don’t have to be poorly written just because most of them are. Stop hiding behind the bullet points of PowerPoint and MBAspeak. A coherent story and cogent ideas do not require the camouflage of paradigm shifts in asynchronous end-to end consumer interfaced concepualisations going forward. If in doubt give your report to a non-marketing person to read. If they can understand it then it will probably make sense when someone who wasn’t at the presentation picks it up later.

The finest research reports I’ve read were from Peter Kenny. Invariably hand written and no more than four A4 pages they were discursive and deceptively simple (impressive as they sprang from a mind so brilliant that only two people on the planet were qualified to grade his doctoral thesis). They usually contained hand drawn pictures and still make colossal sense twenty years later.

Rule 7: Culture matters. Examine the past as well as the present. To understand where societies are headed, look at where they are from. Read Geertz and Mulder’s books on Java and Hla Pe’s “Burma”. Historical, popular and material cultural perspectives are generally more instructive than non-parametric statistics when understanding Asia.

Rule 8: Get out of the office. Research isn’t a white collar occupation although most research companies resemble accountancy firms where researchers imperiously compile data from questionnaires and focus groups that are brought in from the chaotic and non-airconditioned world.

Online communities are great but have not replaced the need for Wandering About. Speak to people on the train, on the street, in their homes, in the bars and coffee shops. Observe. This is a crucial part of the quest for enlightenment, not an optional product clients can buy under the guise of “ethnography”. Keep in mind that you have been commissioned to solve a problem not run “x” number of focus groups or conduct “y” interviews.

Rule 9: Make up your own rules. Read the textbooks but then explore new solutions. For example, the ESOMAR handbook tells us that focus group respondents shouldn’t know each other but, in our experience, it’s preferable to have a groups of teenagers who are friends in order to have them comfortable with their peers. They are also more likely to tell the truth amongst people who know them.

Get out of the traditional group “fishbowl”. Recruit consumers in situ so you know they really do drink XO cognac or go clubbing or whatever your consumers are meant to do. Reconvene groups. Sensitise respondents. Call them up after a group session and clarify their responses in private. Use more than one moderator. Listen unobtrusively. The best researchers are like the best photographers, chameleons whose presence doesn’t influence the experiment.

Rule 10: Enjoy yourself! Research is now cool – it wasn’t always the case. But it was always interesting. The journeymen/women of Asian Strategies have run focus groups in locales as diverse as Melanesian caves (ahh, Plato) and seafood restaurants in Kowloon with triad members who insisted on standing up throughout the two hour session (at least three moderators are required for this as two are inevitably yam singed under the table).

We’ve been poked with spears by irate Papuan respondents, dodged landmines in Khmer Rouge areas of Cambodia, been stoned by drunken fishermen in Banda Aceh and helped extinguish a fire in our hotel on Gizo (Solomon Islands) to save two thousand questionnaires from a radio survey. All in the name of consumer behaviour.

So get into some comfortable clothes, slap on the sunscreen and get out there kids. You probably won’t become a millionaire but you will become a better researcher and be more valued by your clients. Oh, and don’t forget to bill fifty percent of the fee up front.

Cautionary tale 1

Researchers are always telling you how research has improved things. Greg Coops shares an example of “what went wrong” …

After 30 years in research in Asia, I can boast of a longish list of things that haven’t gone precisely to plan. One of my earliest cautionary tales was research for Windsurfer in the 70’s in Sydney. I was trotted out in front of the client as the credible consultant – the firm’s young surfer. Depth interviews were duly conducted on the potential for sail boards in Australia: not as easy as you’d think when the typical surfer’s vocabulary consisted of approximately three adjectives: “cool”,”really cool” and “f#*ked”.

Nevertheless, a consistent story quickly emerged. These boards were too big, too heavy and would only make sense on the lakes of Switzerland where there were no waves but useless in Australia where we had surf and needed only manoeuvrability not propulsion. So I went back to the client and confidently pronounced the idea dead in the water.

Fortunately he didn’t listen to us and the sport boomed and, decades later, morphed into kite surfing. I have been careful listening to “consumers” (sorry) and even industry experts ever since.

Of course what we didn’t predict was (1) that these boards would get lighter and less sailboat-like due to construction technology and better shaping and (2) that a few maniacs would take these sailboards out in the surf. Never underestimate the ability of a young bloke full of testosterone and beer to go beyond what seems sensible and for his friends to follow him out there. Without this our ancestors would never have left Africa but that’s another discussion.

POSTSCRIPT: So where did we go wrong? We forgot to ask people to imagine the ways surfing could be improved and, instead, focussed on testing a prototype as though it was a final product. As obvious now as the New Coke debacle was when the researchers omitted to mention to Coke aficionados that New Coke would replace their current recipe (an error I should add that was not made by the qual. researchers; the quant. report, despite having this flaw, was considered more credible). Oh well. Always easy with hindsight.

Continue reading