SaaS Experiments Podcast: Recent Episodes
Hosted by Brian David Hall
Hear from experts in conversion optimization and growth marketing about what worked, what didn’t, and what’s next.
How do you improve onboarding for a complex product with distinct types of users who have different needs? 😅
Not easy, but it can be done - as explained in this conversation with Product Designer Rafa Studart.
1:59 All the things you can measure with call tracking software
3:01 KPIs
3:47 Forming an internal growth team
4:05 Slightly test-averse culture 😁
4:44 Wasted efforts from launching designs that underperform
5:48 3 months to demonstrate value 😅
6:40 Weekly "war room" meeting + prioritization process
8:07 Starting off with simpler tests
10:03 Biggest issues with customer onboarding
10:29 Multiple valid onboarding paths for different customers
16:14 What the growth team offers the product team
17:34 Initial focus on guidance, rather than explicit onboarding
18:45 After initial success, moved on to more ambitious experiments
19:02 Test concept
20:15 Emphasize the feature most customers need, based on data - even if they don't know they need it yet
20:56 Control version - focused but longggg
23:24 Final results 📈
24:11 That's a huge win 🏆
26:33 Some of the other things we tested
31:35 Options for the small proportion of visitors who don't need a phone number
33:23 Surprisingly few opt outs
34:37 How this experiment changed her approach to design
35:25 Rolling out untested changes and being stuck with them (at least for a while)
36:56 Testing on new users makes things easier
38:03 We took this approach because we had gotten things wrong in the past
38:50 Biggest takeaway
Rafa on LinkedIn: https://www.linkedin.com/in/rstudart/
Rafa's portfolio: http://www.rstudart.com/
Arun Sivashankaran, founder of FunnelEnvy, breaks down all things optimization for Demand Gen. What to measure, where to focus, and where not to focus.
1:13 What makes Demand Gen different from other types of websites
3:48 Quantity vs quality when optimizing for leads vs revenue
6:05 Measuring what matters will change the nature of the experiments you run 🤯
9:48 3 types of changes that drive the most results
17:53 3 common sense buyer journey stages
20:19 Where to start, and how to evaluate your top of funnel offers
23:58 Personalization ... or website retargeting 😎
30:45 The meaning of "friction" in a B2B Demand Gen context
Find Arun here: https://www.funnelenvy.com/
In this episode, I talk to Ben Labay, Managing Director of CX Experimentation at Speero.
He breaks down how to meet the challenge of doing user research when you’re selling a B2B SaaS product to a highly specialized market.
Transcript and other ways to listen here:
https://briandavidhall.com/ben-labay-on-saas-experiments/
2:09 Should you do user testing in B2B SaaS?
4:35 Common user testing platforms & features
5:24 Custom paneling services
7:14 What to look for in user testing
7:51 NPS and benchmarking metrics
10:20 Tradeoffs between platforms with custom panels vs DIY recruitment
12:48 Using industry newsletters for panel recruitment
15:22 Using a tool like Hotjar for panel recruitment
19:16 If you're just starting out ...
21:20 How not to mess up your script and screening questions
26:35 How to detect struggle/friction in user testing sessions
29:42 How many participants do you need to do benchmarking?
31:49 Tradeoffs between highly qualified panelists vs feasibility of finding them
34:25 More on screener questions
40:02 More on pricing tradeoffs
43:22 Avoiding data traps
45:53 "Science is merely a tool to, to try to be fast and confident at the same time in our decisions"
I talked to Andrew Anderson about discovery-based optimization.
Transcript and other ways to listen here: https://briandavidhall.com/andrew-anderson-on-saas-experiments/
The Discipline Based Testing Methodology (https://cxl.com/blog/the-discipline-based-testing-methodology/) – an article on the CXL blog that outlines some of the ideas from this podcast, and includes screenshots from a homepage redesign experiment
Andrew on Twitter (https://twitter.com/antfoodz) and LinkedIn (https://www.linkedin.com/in/foodz/)
I talked to Justin Aronstein from https://mobile1st.com/ about running SEO experiments, and fostering experimentation culture across teams.
Transcript and other ways to listen here: https://briandavidhall.com/justin-aronstein-on-saas-experiments/
0:38 Justin's background
1:13 About FlowSports - business model and goals
3:39 SEO experimentation vs CRO experimentation
6:13 Why run tests if we have best practices?
10:10 Tying SEO updates to revenue
11:15 Set up of the scheme experiment - Bucketing
13:29 Reporting
14:37 More detail about scheme (see link below)
17:51 Success metrics
19:19 Results!
23:24 How results were perceived
24:07 Cultural benefits of running this as an experiment
28:37 Experimentation facilitates team building
29:07 Biggest takeaways
31:08 Engineering & implementation details
38:32 Cultural benefits with respect to the dev team
40:10 What's next for FlowSports & SEO testing
42:47 How to reach Justin
Google schema / content markup guidelines: https://developers.google.com/search/docs/guides/intro-structured-data?
Justin on Medium: https://medium.com/@jaronstein
In this episode, I talk to Viv Amatuzzi, Director of Conversion at 360i.
We talk about the difference between decision makers, product users, and website visitors.
I talked with Tim Duke, Director of Operations at Mobile1st, about the tradeoffs between different calls to action.
Transcript and other ways to listen at https://briandavidhall.com/tim-duke-on-saas-experiments/
Find Tim at https://mobile1st.com
0:56 About the company and product
1:53 About Key Performance Indicators
3:02 The chat/form/call saga starts ... CTA test
4:52 It gets a little more complicated after the first win ... "get more chats"
8:16 "We can't just make everybody chat" ➡️ "Three-card Monte style CTAs"
10:17 ... leads went up across the board
11:53 Increased chats ... ignored emails 🤦♂️
13:09 "What are you really good at?"
13:56 "Cater to your strengths, and focus on your time to close"
16:24 It's about how you build out your org, not how big the button is
18:31 Advice to a CRO dealing with competing teams
20:00 How did this series of tests change your approach to UX and CRO?
21:09 "Optimization needs a seat at the biggest table"
23:36 What do you think about automated chat 🤖 ?