#Conversion2015 Amsterdam keynote Ton Wesseling
•
3 likes•4,066 views
April 14th - 09:45 - 12 hours earlier Ton Wesseling was asked to replace Craig Sullivan as morning keynote of Emerce Conversion 2015. Based on the ROAR model he created for CROday 2015 and the analytics tips on A/B-testing slidedeck he created voor DDTT on A/B-testing, he presented these slides for a 350 people audience in Amsterdam. Any questions? Email: ton@onlinedialogue.com Twitter: @tonw
Recommended
Recommended
More Related Content
What's hot
What's hot (16)
Viewers also liked
Viewers also liked (9)
Similar to #Conversion2015 Amsterdam keynote Ton Wesseling
Similar to #Conversion2015 Amsterdam keynote Ton Wesseling (20)
More from Online Dialogue
More from Online Dialogue (20)
Recently uploaded
Recently uploaded (20)
#Conversion2015 Amsterdam keynote Ton Wesseling
- 1. ONLINE DIALOGU E G u i d i n g y o u t o d i g i t a l m a t u r i t y D e m y t h e n , d e l e u g e n s e n d e i l l u s i e s # C o n v e r s i o n 2 0 1 5 – A m s t e r d a m – A p r i l 1 4 t h 2 0 1 5 To n W e s s e l i n g A /B-testen
- 2. Email: ton@wesseling. net! Linkedin: t on wesseling! Tw it ter: @tonw! ! Ton Wesseling!
- 3. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Webanalisten.nl
- 4. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Online Dialogue
- 5. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Testing.agency
- 6. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Emerce magazine: conversion column
- 7. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Conversion Conferences worldwide
- 8. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Conversion speakers
- 9. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Where is Craig Sullivan?
- 10. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Replacing Craig…
- 11. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 At least he quotes me
- 12. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So..
- 13. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More experiments “If you double the number of experiments you do per year you’re going to double your inventiveness” Jeff Bezos, CEO Amazon – 2004!
- 14. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Like Amazon, Like Zalando GermanEcommercemarket,thankyouAndréMorys
- 15. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Experiments per year 1000+
- 16. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015
- 17. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why not? Compare to the past?
- 18. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why can this lead to failures?
- 19. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 A big weekly change! 26 out of 44 weeks: over 5% change!
- 20. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Methods used to improve conversion
- 21. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Methods companies plan to use
- 22. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s just HOT
- 23. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Lack of resources
- 24. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Test more
- 25. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Importance of a structured approach
- 26. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More experiments
- 27. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 The negative effect trend Time span! Averageeffectofoptimzation! atcompaniesapplyingoptimization! Moneyspendon! Optimizationworldwide!
- 28. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s not a solution It’s a method
- 29. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 It’s a way of working It’s DNA
- 30. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So it’s a method: When How What
- 31. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Growth curve – how it was teached
- 32. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Optimization phases - ROAR Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 33. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Phase 1: bridge the gap – Risk
- 34. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake adwords testing Do they click on your idea?
- 35. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake facebook testing Do they like your idea?
- 36. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Fake landing page testing Do they subscribe to your idea?
- 37. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Phase 1: bridge the gap – Risk
- 38. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Get out of the office
- 39. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Risk
- 40. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR - Optimization Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 41. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: low significance levels ü Big chance the winner is not a real winner
- 42. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: low power levels ü Big chance a real winner is not recognized
- 43. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Example test set-up ü 90% significance level ü 100 conversions per variation ü Average conversion rate: 2% ü 20 out of 100 ideas is a winner… ü …with an average uplift of 10%
- 44. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Average: 75% significance & 40% power
- 45. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So: ü 100 tests will give 10 false positives (90% significance) ü 40% of the 20 winners are recognized(40% power) ü 8 real winners & 10 false positives ü NOT 18 improvements of 10%+
- 46. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You need big numbers per variation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 47. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 With 1000 conversions + a month 20+ tests a year
- 48. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tests with impact, on average 1 out of 3 Simply truth: you’re just not always able to create a winner with enough impact!
- 49. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So in this case, you’ve impact Every 6 weeks
- 50. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Slower? Energy will run out!
- 51. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Business cases! Delivery –test with in stock. 4 weeks or 2 days?!
- 52. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Business cases! Service charge, do it or not? Include in price or not?!
- 53. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 What to test? Big design changes
- 54. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Warning: choose what to analyze up front ü You just go and dig unitil you find something (you will always find something that seems true BUT is not!)
- 55. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Don’t stop the test once it’s significant!
- 56. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Don’t stop the test once it’s significant!
- 57. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You just let it run until it’s significant
- 58. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You just let it run until it’s significant Your test has a fixed lenght! It’s even a fixed amount of test population, but we love to test business cycles and/or weeks!
- 59. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ABTestGuide.com/calc
- 60. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Optimization – moving up Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 61. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Finally you can test details… Is Multi Variatie Testing (which is also A/B-testing but with more varations) a good idea? Almost never!!
- 62. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You should grow to Customer Buying Reasons
- 63. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Reasons behind the customer journey
- 64. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Why is your user here?
- 65. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 How can you help your user’s brain to fullfil it’s needs?
- 66. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Data & Psychology Digital Data Persuasion Psychology Analyze & Experiment
- 67. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Testing takes time Use it to LEARN
- 68. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Lack of resources
- 69. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Create a test schedule
- 70. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 FA C T & A C T – 7 s t e p s t o l e a r n & Tell Conclude Analyze Test Create Analyze Find
- 71. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Psychology
- 72. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Ask
- 73. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Source: Listen
- 74. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Many sources out there to find & analyze
- 75. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Test which dialogue has impact
- 76. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Analytics
- 77. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Heatmaps
- 78. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Which locations are critical
- 79. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 FA C T & A C T – 7 s t e p s t o l e a r n & Tell Conclude Analyze Test Create Analyze Find
- 80. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now you’re testing On your way to 4 tests a week Max capacity with 1 teammember on each discipline: analyst, ux, developer, psychologyst!
- 81. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Your winners are still spare? Less than 25%? Please higher your significance level to 95% or more if it drops below this 25 number!!
- 82. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Your test takes too long ü Cookie deletion (you will loose 10% in two weeks) They will re-enter the the and pollute your samples!
- 83. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Don’t confuse cross device users ü Only test 1 device at a time!
- 84. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: Maybe you’re testing broken stuff ü Always check browser compatibility! ü Dont’ break dynamic stuff!
- 85. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You forgot to leave out some visitors ü Only test on those who can be changed (if you test a sales promotion – leave out your current clients who just logon)
- 86. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You’re not slowing the control down ü Add the same code to the control
- 87. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You’re not testing on fresh people ü Start with control for a full purchase cycle (and then start sending traffic to your variations – setting the control to 0%)
- 88. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Tip: You forgot to give them time to buy ü Stop getting fresh visitors after your test time (send new traffic to the control, but give you tested people time to finish converting)
- 89. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 90. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You’re already thinking of more teams And still have test capacity *test capacity – the number of tests you can start per week based on your audience & conversions!
- 91. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Algorithms
- 92. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 You tested which locations are critical
- 93. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Automation
- 94. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now you’ve moved to Algorithm testing Which internal and external influencers do have an impact on which segment of users?!
- 95. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – Automation Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 96. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 CRO becomes user knowledge
- 97. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 More impact on discover
- 98. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 A way of life
- 99. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Customer knowledge center Making money from conversion improvements Creating more and more customer insights Which can be monetized on even a larger scale
- 100. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Now, that’s HOT
- 101. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 So you keep on groing GermanEcommercemarket,thankyouAndréMorys
- 102. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 Done testing? No! TESTING NEVER STOPS Okay, end of life, cash cow, no need for any further growth accelaration or user knowledge!
- 103. onlinedialogue.com #Conversion2015 – Amsterdam – Ton Wesseling – April 14th 2015 ROAR – when to start with what Time span! Conversionspermonth! Risk! + Optimization! + Automation! Re-think! 1.000 conversions! per month! 10.000 conversions per month!
- 104. ONLINE DIALOGU E G u i d i n g y o u t o d i g i t a l m a t u r i t y D e m y t h e n , d e l e u g e n s e n d e i l l u s i e s # C o n v e r s i o n 2 0 1 5 – A m s t e r d a m – A p r i l 1 4 t h 2 0 1 5 To n W e s s e l i n g A /B-testen
- 105. Email: ton@wesseling. net! Linkedin: t on wesseling! Tw it ter: @tonw! ! Ton Wesseling!