Client profile
- b2c ecommerce
- vast majority of traffic obtained through media buys (large amounts of poorly-qualified traffic)
- 97% of converters purchased on their first visit
My role: I coordinated and led this in-house effort.
The challenge
A very well-established conversion funnel hadn’t shown any dramatic improvements despite months of tweaking and testing. We needed to try a new approach.
I stole the Hackathon idea from Google or some other buzzy tech company (maybe it was Facebook’s Hack-a-month?).
The process
I assembled a cross-disciplinary team to brainstorm solutions. I brought in two back-end developers, a graphic designer/UI expert, a media buyer, and a senior member of the customer service team. We hashed out possibilities and zeroed in on what seemed to be an achievable goal.
Biggest opportunity: Mobile traffic had always converted below desktop visitors, so we decided to fix that.
Having decided on a target, we pulled in another handful of staff for an epic brainstorming session. Because I wanted a from-scratch approach, I encouraged total freedom and even outright silliness (I brought in a chilled case of energy drinks, a deck of tarot cards, and my set of Rory’s story cubes to help shake loose new and different ideas).
After a couple of hours, we moved from “yes and” to “yes but” and short-listed fifty optimizations. Then we prioritized each based on time-to-live and likelihood of success. We selected the 14 most achievable and high-impact concepts (an aggressive goal — 14 tests in 14 days!).
The team chose to focus on:
- Usability
- Text size
- Copy length and placement
- Touch target size
- Review of mobile device screen sizes and resolution
- Interactivity
- Real-time comments from visitors on the site
- Script-generated procedural comments directed to visitors
- Impact
- Increase size of key graphical elements
- Experiment with size and placement of graphical calls-to-action
- Friction
- Deliver nudges at key moments during the conversion process
- Reduce the number of fields customers fill out
- Reduce the number of clicks required for conversion
Time to get to work.
I directed all Hackathon team members to set aside all but the most critical of their daily tasks and move into a shared workspace. I had lunches and snacks delivered. I posted warning notices and do-not-disturb signs and implemented the soon-to-become-notorious Hat Rule:
If someone is wearing a hat, that means they’re thinking. Do not speak to anyone wearing a hat.
Aside: This might seem like a really dumb idea. Here’s the thing: it totally worked. In start-up environments, key professionals frequently get pulled in different directions — whether it’s the head of IT fixing a problematic printer or the senior logistics coordinator answering the phone because the receptionist is at lunch. And that’s awesome we’re all willing to pitch in to get the job done. On the other hand it gets in the way of focused, deliberate thought and action. The hats? They informed the rest of the staff that the teammate wandering around looking thoughtful in my dad’s old Vietnam-era M1 helmet was actually working. The shared workspace, endless energy drinks, and Go Away signs helped gel the Hackathon.
By the end of the first day, we were able to complete a handful of variants and begin testing. We followed the rules of !!science!!
- each variant contained a single change
- traffic randomly assigned between variants
- page versions each contained a unique tag that enabled us to track visits throughout the conversion funnel
- tests concluded when a two-tailed Z test for statistical significance reached 95%
Results
At the end of 10 working days:
- Android conversion rate: +19.5% (99% confidence)
- iPhone conversion rate: +18.7% (95% confidence)
- +$75,000 revenue per month without additional costs
Combining best-of-breed solutions squeezed out even more performance. However, the Hackathon was over for now. (The owner was so impressed he decided to make this a quarterly event.)