Marketers' Breakfast: Experimentation

29th Sept 2017
Thursday 28th September
The Anthologist, 58 Gresham Street, London

For the latest instalment of the Amigo breakfast series, we invited marketers from a range of industries to discuss experimentation.

Here are some highlights from our conversation.


Naturally, we began by talking about data. A huge and sometimes controversial topic for marketers.

David Lim, Consumer Marketing Leader at Mercer, mentioned Spotify’s billboards, which use listener data to tell stories about current events. Data can often seem dry and abstract, so these are a great example of connecting it to real, emotive events.

Other examples mentioned include how data analysis is used by political campaigns. Chris Hadley, Senior Data Scientist at Dunnhumby, and Rosie Parkin, Lead Digital Analyst at Save the Children UK, both mentioned Cambridge Analytica’s work with the 2016 Trump campaign, as well as the 2008 Obama campaign’s revolutionary use of micro-donations and digital targeting.

Many marketers currently see data as paramount. As Colin Smith, Co-founder of Cambridge Data, noted, even the insurance giant Aviva is attempting to present itself as a tech company, which uses its telematics app to drive down premiums. However, while marketers are comfortable with the allure of data analysis as a branding concept, actually using data for practical marketing purposes often poses rather more challenges.


Siloed data, which we have analysed as a key component of the marketing execution gap, continues to hold marketers back. Peter Kent, Head of Digital Marketing at Investec, stressed this point and described how he was attempting to bring disparate CRM data together at his own organisation. Almost all of our guests could relate to dealing with too many data sources and the problems this posed for experimentation.


Existing data cannot be the only starting point for experiments. Jake Mellett, the founder of Joeywears, a men’s apparel company recently funded by a successful Kickstarter, noted the problem of digitally-native marketers’ overwhelming focus on data. Hamish Rickman, an experienced Global Marketing Director, agreed with Jake’s advice to conduct focus groups before analysing data, proposing that “if you haven’t spoken to your customer, it doesn’t actually matter what colour your CTA buttons are.”

Jake continued to explain how early in his career he became almost addicted to plugin tech for conversion rate optimization, such as Optimizely, something which we have mentioned before on our blog. In his experience, frantically making small iterative changes did little to generate real commercial results.

In support of Jake’s point, Aimee Wilde, one of our Account Managers at Amigo, recalled her recent experience at the Figaro Digital conference. A marketer from Unicef had complained about the relentless focus on optimisation through iterative changes because it had blunted the marketing team’s ability to envision and deliver large-scale, creative campaigns. There was a common sentiment that large campaigns are more likely to deliver much higher returns in the long-term, if only marketing departments would have the patience.


The question of whether organisations have the patience for experimentation is part of a larger problem of risk aversion. As Hamish Rickman put it, “if you don’t take risks, everyone dies very slowly.” In the long-term, David Lim noted, there needs to be patience and trust from the C-suite for a culture of experimentation to thrive.

Rosie explained how this could be even more difficult in the charity sector, where accountability for marketing spend has to be much higher than in businesses. However, she argued that it was essential for charities to experiment in order to minimise the risk of spending money unwisely.

Fear of Failure

Peter Kent provided a great example of a means to overcome risk aversion by becoming the failure-tolerant leader for his team. At Investec, he introduced a KPI for failed experiments. Peter’s team are freed from the fear of failure by being required to fail against at least one of their targets in order to receive a successful quarterly review.

To support this point, Frederic Kalinke, Managing Director of Amigo, mentioned the book Creativity Inc., by Ed Catmull, a founder of Pixar. Catmull explains the importance of being candid about failure and suggests that a “Braintrust” is a good way to promote the right culture in an organisation.

Starting Small

Another way to deal with risk aversion is to start small. Carl-Henric Heimdal, a Brand and Growth Manager at Swiss Clinic, demonstrated his experience of exactly this approach. Swiss Clinic used Sweden as a test environment for launching their business. This allowed them to establish a number of successful tactics, such as serving products as answers to Google search queries about skincare, before scaling up. As Frederic mentioned, this approach has also been successful in completely different industries. The payroll company Xero carried out a similar strategy in New Zealand.


Another necessary condition for marketing experimentation is that experiments become easy to carry out. Colin in particular focussed our discussion on the internal processes that stifle marketing experiments. He recalled speaking to a Head of Analytics at a Pensions and Insurance firm who stressed the importance of experimentation but could not tell him what percentage of their marketing campaigns were actually undergoing any testing.


Rosie added that marketing needs to look to experiment everywhere it can, rather than just with simple variations on campaigns. In her experience, this has meant looking at new ways to target and segment customers. Rosie claimed that charities are too often stuck appealing to one donor group (‘middle England’ woman) and are missing out on opportunities elsewhere. Frederic concurred, noting the example of charity: water and their innovative approach to marketing.

How to Experiment

A few themes recurred throughout our discussion and some crucial pieces of advice emerged.

Marketers who want to conduct experiments need to do the following:

  • Ask the customer before staring into the data
  • Have patience and take a long-term view
  • Embrace failure as an opportunity to learn
  • Break down data siloes

Today’s breakfast demonstrated the importance of running experiments in marketing. However it also highlighted the difficulties in creating an environment where experiments can work. Enterprises have to have the right culture, skills and technology. The culture needs to promote risk-taking and celebrate failed experiments as much as successful ones. Marketing teams need to become digitally savvy across the board and marketing technology needs to work with rather than against existing legacy systems.

photos by Gyan Gurung

If you’re interested in attending an Amigo Marketers’ Breakfast, then please do get in touch.

Further reading

020 3940 4650