Occam: Recent Episodes
None
The intelligence and automation revolution has created/forced immense transformation at Marketing Agencies.
All types: Media. Creative. Performance. Brand. Measurement. CRM/”Lifecycle.” Advocacy. Auditing. AI. Combo of all.
Three strategic drivers of the we are not in Kansas anymore realization:
- AI achieved broader, more general intelligence.
- Ad platforms happen to own foundational AI models. Translation: Intelligence became available, automated, and actually useful FAST.
- Interconnectedness of systems. Data moves in real-time across your stack, platforms, partners, everyone. That doesn’t make AI smarter in general. It makes AI smarter about you.
These delicious drivers of behind the scenes changes at Agencies are now ready for a cascade of benefits for Clients.
More, For Less. Really!
Immediate and medium-term implications:
A. The reduction in work currently listed in your Agency SOW will translate into a 25% – 75% savings in Agency fees, starting July 2026. For all types of Agencies listed above.
B. New work currently underpowered or unlisted in your Agency contract will translate into a 15% – 25% increase in Agency fees.
A+B will save money AND deliver materially better business outcomes AND make change-embracing Agencies indispensable deep partners.
C. Previous TMAI editions have shared the scale of change in Performance Marketing. These contracts will shrink 75% – 80% re work/fees. I’ve also shared the increased strategic importance of Brand Marketing. These contracts will shrink 25% – 40% (due to A), while overall budget will likely go up, possibly a lot (due to B).
[Premium Subscribers: Please dive back into TMAI #496: What The Heck is Brand Marketing – the implications will power the next 10 years of your career. Email me if you can’t find it.]
Change will happen on sliding scale. Ex: Everyone can get A-driven 25% to 35% savings from work elimination today. Achieve about the same reduction next year – at that time you might also add 10% B-driven. So on, and so forth.
This post was first published as TMAI Premium #516. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff — built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.Cut to Grow. Really?
My objective is not to save on Agency fees. It is to create incentives to fully embrace the present and be ready for an unknown future.
If your Agency is working off an old-world Statement of Work (SOW), all the incentives, legally, aligned to keeping the past as the future.
Current contract has no incentive to shift work to platform intelligence and automation – thus reducing Agency hours. The SOW rewards manual work: micro-optimization, reporting, complex campaign setup, etc. – all harmful to AI-led efforts. Your contract’s Percent of Media Spend rewards spending. More and more and more activity – easier via non-AI! Paying using KPI Percent of Outcome creates AI-Impact alignment – it is not in the contract.
Renegotiate your Agency contract not to save money.
Do it to:
A. Align Agency’s incentives to embrace AI and Agentic everything, codify doing less can be more by paying big for Outcomes.
B. Incent the Agency to invest in building skills in areas AI and you can’t solve for (more on this below).
C. Add new big fee line items for items critical today and existential in 2027.
Critical lens on Agency fees is simply a mechanism to accomplish A, B, C.
When you renegotiate, you are not buying fewer hours of the same old work. You are buying a different operating model.
A Gigantic Caution.
Nothing I’m recommending will actually work while the Agency fees is a Percent of Media Spend.
That model literally pays the Agency to touch more, spend more, and be anti-AI.
To make the present the future, your Agency contract structure needs to migrate to:
1. A lean base retainer for governance + steering + data engineering. Approx. 40%-50% of the new, smaller total.
2. Project fees for creative concepts & pre-testing, complex strategic analytics (not “measurement” or “reporting”), and portfolio strategy. Approx. 30% – 40%.
3. An outcome incentive tied to incremental profit (or at least incremental revenue, verified lift, and NEVER platform ROAS). Approx. 15%-25%.
The Agency will now offer genuine specialist depth, cross-client learnings that power speed of innovation, cross-platform OEP consumer behavior informed strategy creation, and new outside-in scale… Or additional fees reductions become possible.
Everybody Wins, YES!
If you lead an Agency, I’ve likely added to your discomfort. Let me make the case for why this change is incredibly exciting for Agencies.
A. You are no longer a hamster furiously rotating a wheel connected to nothing.
Agency gets to work on really cool, smart, high-value, unquestionably outcome-impacting initiatives. This extends the life of the Agency. This enriches the careers of every Agency employee.
B. You don’t need to be doing $50/hr or $100/hr work.
Your new SOW contains more expensive $500/hr and $1,000/hr work streams, along with $yyy,yyy per week costing strategic projects!
Glorious impact: This new and exciting work allows you to hire large numbers of entirely different types of experienced individuals – while paying them well (vs. the current: charge the client a lot, hire the most junior employees you can get away with, and pay them below market wages in exchange for “experience”).
C. (As with every revolution…) Better Agencies will emerge.
Agencies have died and were born each day last decade. I anticipate both accelerating in the next two years.
Births will be so exciting. Modern Agencies attuned to persuasion in an AI world, responsive to shifting consumer experience, embracing/leading radical evolution of platforms, structurally built to be outcomes-centered strategic partners! Mamma mia!!
The Core Shift.
The old Agency fee was largely rent on an execution army: campaign builds, keyword lists, audience segmentation, manual bidding, painful upfront contract negotiations for the next 18 months, pacing, trafficking, QA, reporting, and constant “micro-optimization” on ad platform to feed Clients with the illusion of “work” – and have something to show in the 2x/wk meetings Client employees demanded to “prove value.”
Except for the illusion need, most levers above are pulled by the platform’s own AI for Performance Marketing and many for Brand Marketing. You’ve heard the names: Smart+, ASC, PMax, AI Max, yada, yada, yada.
[Premium Subscribers: Calculate your sophistication score: TMAI #508: Google Ads Maturity Model. +Discover what your Agency should have been doing mid-2025 onwards.]
An Agency’s value has migrated upstream: Cross-platform OEP consumer behavior informed strategy creation. Value signals hunting and automation. Creative concept ideation and pre-testing (execution and variations are AI helped or solved). Super advanced holistic full-anywhere-outcomes analytics. Governance (guardrails, Agentic wrangling, brand safety). Some or all of these create new client impact and new client fees.
The Agency should not be paid primarily for touching the account.* In fact, over-touching now actively degrades algorithmic performance by messing up AI learning!
The Agency should be paid for automation, more senior judgment, governance, scaling intelligence, and bleeding edge innovation risk reduction (not “A/B Experimentation”).
Agencies are not becoming worthless. The basis of their value is materially changing.
Exactly as, Analysts / Creative Directors / Content Creators are not becoming worthless. The basis of their value is materially changing.
- If in July 2026 your Agency is touching your account more than once every dozen so days, do not blame them… Fix the source by firing your Marketing Manager & Director.
Subtractions & Additions to Power Your Shift.
As someone with leadership roles at Clients (15+ years), Ad Platforms (15+ years), working closely with Agencies (10+) and leading one (2+), I say with confidence that the future is super bright for Agencies with good CEOs.
To support that transformation, presenting my… Client < > Agency Operating Model.
This week: Subtractions. Everything transitioned to Platform AI or reduced or eliminated entirely, resulting in changed Agency scope and resulting contract savings.
Next week’s Premium edition: Additions. A. Everything your team should own now. B. Streams of higher value work the Agency needs to grow into and be paid more.
Subtractions: What You Stop Paying For.
Extracting from a typical past Agency contract, the subtractions contain twelve dimensions to focus on. For each, I’ve identified the typical cost weight, reduction in Agency effort, and contract savings.
For each of the 12 dimensions, in a robust spreadsheet, I’ve identified: Area of Work. Old Model (what an agency did). New Model (powered by platform AI). New Company Role. New Agency Role. Change in Agency Scope. Estimated Contract Savings.
Since my experience along all of these dimensions would take a mini-book to detail, I’ll compress 12 dimensions into 5 clusters and summarize your actions.
[Note: TMAI Premium subscribers can reach out for the robust spreadsheet to direct the conversation with their Senior Leadership, Procurement, and CFO.]
- Agency Activity Army.
Account and campaign architecture, “keyword” research and match-type sculpting, audience segmentation and targeting, radio & tv ad tactic structures. This was the beautiful old world of a hundred thinly sliced campaigns “for control.”
The machines devoured this work. PMax, Advantage+ (A+), similar collapse structure into a few asset groups that the algorithm allocates internally. Intelligence reads intent, using additional tens of thousands of data points beyond the keyword or post content. A+ audiences treat your segments as hints, and then finds converts you never listed.
The Agency’s job is not building, it is deciding. Advertiser (you) sets the reward function hierarchy, brand/non-brand priorities, brand safety red lines, margin protection bands. The Agency’s real remaining role: One-time architecture design, and occasional restructure IF strategy changes. It is NOT monthly rebuild billed as progress.
Approx. 22% contract cost weight, work that can be reduced by approx. 78%.
- The Bid & Pace Dancers.
Manual bids, budget adjustments, day parting, device modifiers, daily pacing, spend checks, disapproval fixes, “anomaly” checks, “hygiene.” Hours and hours and hours.
AI can do narrow intelligence at a scale and impact that surpassed us in late 2024.
Smart bidding to your reward function (see: TMAI #432: AI Unlock: Value Based Bidding), campaign budget optimization, automated alerts, auto-recovery, now all this work continuously, and better. You just need to let the AI learn. When it starts, results will dip a bit. Your Agency jumps in to rescue by cutting bids/something. The AI learning resets. The dip returns. Agency… You know the rest. No stable learning. Company keeps sucking.
Reminder: Over-touching does immense damage in an AI world. Your Agency is causing this harm and it is your fault – for not understanding that every “rescue” is a sabotage.
The Agency job shrinks to setting the true reward function (aka kpi, target), occasional configuration of guardrails IF business strategy changes, and, hardest of all, operate at the rhythm of the AI’s learning cycle.
Approx. 14% contract cost weight, work that can be reduced by approx. 73%.
- The Assembly Line.
Trafficking, ad builds, combination making, ad formats \/ creatives \/ variations, torture of tagging, QA, shopping-feed babysitting.
Ad platforms – except TV, Radio, press – can do this at a scale AND relevance that will shock you. Assembling bits from your ads, text, site, building creative, generating variants with GenAI (try this on paid Amazon!) mixing image video audio in the best version per person. All they need: often just your site link (or core assets) and a reward function.
Note: One reason they can do this spectacularly well is that your creative is judged by your VP of Creative/HiPPO, while the creative Ad Platforms deliver is judged by the business outcome delivered. 🤯
The Agency’s role narrows to the genuinely useful: Asset preparation, clean taxonomy, and spot-QA. Feed management is the one line that remains meaningfully skilled for now – sadly, data quality is still human.
Approx. 12% contract cost weight, work that can be reduced by approx. 45%.
- The Optimization Theater.
Daily rituals of pausing “losers,” shifting budgets by feel, endless small A/B tweaks, the deeply irritating unfocused “learning agenda” items that will never amount to much.
Often the single biggest cost, and now the most destructive thing your agency is doing.
Modern AI-led platforms run their own continuous explore-exploit; they are already testing combos of creatives, bids, targeting to find high-value customers – at scale and cray cray speeds. They can also account for learning cycles that can span weeks/months, try that with a human.
The Agency’s role is cut down to: Few, really big (as in 15%+ increase in revenue), clean experiments, with the right altitude (creative concept – not execution -, audience attribute).
Approx. 16% contract cost weight, work that can be reduced by approx. 75%.
[Premium Subscribers: The role of what an Analyst does is quickly changing. Your current role will be gone by Jan 2028, replaced by a better one. If you are willing to learn and change, here’s how to save your career: TMAI #495: Analyst 2028: S.H.I.F.T For Relevance.]
- The Reporting & Servicing Factory.
Manually pulled weekly decks/spreadsheets/update emails. 2x Weekly “update” “check-ins” with 6-36 Agency attendees. Exhaustive reports on placements and “brand safety.” Agency “account management” meetings with other vendors and partners. Detailed hand written commentary of the data already in dashboards. Urgent need it by EOD cuts of existing decks/reports. “Client Training Sessions,” with content already easily available to all anyway. Dinners, outings, offsites, to ensure client happiness.
AI cannot solve for all these items, certainly not dinners and outings.
But, dashboards and reports are disappearing with Claude-fronted data lakes that can do both what and why to such a great extent that you don’t have to be held hostage by the Agency’s 17-tab spreadsheet. Platforms for automated placement filters, and brand safety tooling that gets better by the hour. Since there is ever increasing automation in Meta and Google, you need drastically less “account management,” and certainly not as many meetings.
The Agency’s work is limited to: Aforementioned dinners. Automating in-flight optimization. Shifting focus from Care (99% today) to Do & Impact (TMAI #391: Data Storytelling Framework: Care, Do, Impact).
Approx. 30% contract cost weight, work that can be reduced by approx. 60%.
Punchline: Contract Cost Weight * Reduction in Work and in my case equals 65% savings. For you, it might be a little less or a little more. Use the table and the math above to discover.
The work is not going away, a lot of it is moving to machines. What’s left is more useful, higher impact work. Agencies don’t sell motion anymore. They sell judgement.
The Conversation Continues in Part 2.
Achieving this transformation in our Agentic AI-era requires you add new roles to your company to thrive in this new universe.
You also need to add/increase SEVEN threads of new work to your Agency contract.
In Part 2, I’ve covered both these critical elements in detail. Additionally, it also contains the complete Operating Model in Excel, to simplify transformation conversations with your CMO, CFO, VP Procurement… And then, your Agency.
If you are a TMAI Premium subscriber, please reach out for Part 2 if you are unable to locate it in your inbox.
Bottom line.
I come to Agencies from a place of respect. The best ones are extraordinary – taste, courage, and pattern memory across dozens of clients that no in-house team can match.
I come to AI from a place of trust. A vast majority of performance execution is AI-led, and all of it will be soon enough.
That respect and trust sit with this pragmatic truth: You can pay for the past or you can embrace and extend the present.
A new tomorrow starts with knowledge, and now you have it.
Carpe diem.
PS: A little in the weeds but…
A. With the extraordinary criticality of data and automation, it is important that you own your ad accounts, pixels, and any data Agency is piping from sources beyond you. Avoid becoming a hostage or victim. Get immediate and complete ownership.
B. When you kill Percent of Media Spend, you are putting a kibosh on toxicity of undisclosed markups, principal media, and rebates. Awesome. Be aware: There may be people inside your own company for whom that toxicity is a positive incentive. Don’t forget to fix it.
The post Pay Less, Grow More: Agencies in an Agentic AI-Era. appeared first on Occam's Razor by Avinash Kaushik.
In a world with fuel-efficient long-range motorcycles, do you insist on using horses for transportation?
I’m of course talking about the Google Ads platform.
Like it or not, Google has been AIfying their platform for some time. [Meta as well.] They are making it harder and harder for us to stay with how it used to be. Not such a bad thing, because to achieve better outcomes we can do less and the AI can do more.
Hence, I admit to immense frustration when in meetings and online discussions, “Search marketing” is still framed as brand, non-brand. Search “Marketers” are still monekying with match types, hours, geos, audiences, negatives, and all the rest. Search “Agencies” are still proving their worth by making changes multiple times a day. All this is going backwards, profit pulverizing.
☹️
I want to share two biases:
1. I spent 16 years at Google. I worked on projects with the Product and Engineering teams – one of which won our team the rare Google Founders Award. Through my work, I come to Google from a place of trust. Sure, Google & Googlers make mistakes and doozy decisions at times. Most are explained by Hanlon’s Razor.
2. I come to AI from a place of trust. I believe the vast majority (all of Performance) will be powered by AI. Hence, I believe the optimal strategy is embrace and extend. Sure, AI makes mistakes and doozy decisions. But it learns, it improves, every day!
You might come to Google and AI from a different place. I respect that. In a business context, consider this pragmatic perspective: The choice to stay in the past is not available.
So… Why lose our employer cash every single hour? Why put your career in reverse gear? Let’s embrace the possibilities today and extend our winning streak!
This post was first published as TMAI Premium #507. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff — built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.Current AI Offerings in Google Ads.
Google’s “let us do everything for you everywhere” campaign is called PMax. Give Google your goals, a budget, a bunch of assets (text, images, videos, logos), and Google’s AI then mixes and matches these assets to deliver ads across Search, YouTube, Display sites, Gmail & Maps.
Google’s AI-powered Search-only campaign is called AI Max. No concept of brand, non-brand, match type, etc. AI Max assesses intent in real time and promises to deliver your ads to highly relevant search queries. The key bit: It looks beyond search keywords typed, and analyzes intent to expand your audience.
Demand Gen is a visual option (aka ex-Discovery ads) that uses AI to assess intent and deliver your video ads, Shorts, and other ads.
Two things are happening with all three: A. You control the “reward function.” AKA: What winning means for you. B. AI assesses intent across a significantly wider set of signals than you and I can imagine.
PMax, AI Max, and Demand Gen are not perfect yet. Still, as applications of narrow AI, they are already better than us humans, and getting better every day. Losing three or four times while winning thirty or forty times is a decent tradeoff.
Maturity Assessment Killer Parameters.Are you truly living in the AI present, or are you a tourist in the present?
To help you do a genuine assessment of horse riding or motorcycle riding, I’ve built a simple maturity model. The self-reflection is powered by two dimensions.
Capability Scoring: How sophisticated are we?
You’ll score your company on a 0–4 scale in each dimension.
0 = Legacy.
1 = Mostly legacy with some automation.
2 = Hybrid.
3 = Modern.
4 = AI-native.
Depth Scoring: How widespread is our sophistication?
I’ve included this to help prevent self-deception. We will use relevant spend / conversions / campaigns to assess how widespread.
0 = None.
1 = Pilot only, under 10%.
2 = Partial, 10–39%.
3 = Scaled, 40–74%.
4 = Execution default, 75–100%.
To prevent self-deception, I recommend:
- If your depth is 0 or 1, capability cannot score above 2. Admit you are not really real.
- If depth is 2, capability cannot score above 3.
- Only depth of 3 or 4 can earn a capability 4. My model will score your maturity in two steps:
Step 1: Dimension Score = Weight x (Capability/4) x (Depth/4)
Step 2: Company Maturity Score = Sum of Six Dimension Scores.
Your destination? A score of 85 or higher.
Google Ads Maturity Model | Six Dimensions.
Our objective: Nearly all business outcomes are optimized by Google, using automated modern intent <> value signals.
The Google Ads ecosystem is complex enough, diverse enough, that you can take many different approaches. Mine is clustered and prioritized based on my experience and what I see a couple years out. Here are my six dimensions:
1. Measurement & Value Architecture.
Weight: 30 Points!
Because: If you don’t know where you are going, any road will take you there – and you’ll be miserable at the destination.
Google’s intelligence and automation can only optimize to the signals (reward function details) it receives. Enhanced conversions improve measurement and unlock stronger bidding; data-driven attribution is the default for most conversion actions; value-based bidding is explicitly about maximizing conversion value such as revenue, profit, or lead score; and Data Manager is now Google’s central layer for activating 1P and offline data.
The one thing you still control is the reward function – what the AI should deliver that’s of value to you. Hence, if your measurement and value architecture is weak, everything above it is built on sand.
> > How to score Capability?
0: Your team is optimizing for lame proxies such as pageviews, sessions, basic leads, or “all conversions.” No enhanced conversions. No offline data.
1: Basic online conversion tracking exists, but it still treats every lead or sale as equal.
2: Macro conversions are cleaner, enhanced conversions are live, but value is still crude or incomplete.
3: Revenue or lead-quality values are passed back to Google. Offline conversion imports or CRM feedback exist. DDA is in place.
4: Bidding is driven by true biz value: Revenue, profit, closed-won value, predicted LTV, etc.
Score from 0 to 4, see legend above.
Horse rider tell: “We trust automation, but we still optimize for traffic.”
Motorcycle rider tell: “We let automation optimize to our reward function of what a good customer is.”
> > How to score Depth?
Truth Question: What percentage of business outcomes from Google are measured by modern value signals?
Evaluate depth using share of conversion value or lead volume covered by modern measurement:
% of macro conversions using enhanced conversions.
% of conversions tied to value (ex: revenue, LTV).
% of leads/sales with offline or CRM feedback loop.
0 to 4, see legend above.
Great News: In TMAI #508, I’ve shared my maturity assessment as a working Excel model. Once you punch in your Capability and Depth scores, it will provide a personalized set of actions to bring your company into the Ads intelligence and automation age! If you are a TMAI Premium subscriber, please email me for it.
> > Actions to arrive at the present?
For the measurement and value architecture my model will select from:
- Fix primary conversion actions.
- Enable enhanced conversions.
- Import offline or CRM outcomes.
- Move from volume-based bidding to value-based bidding.
A company can have good creative, tidy campaigns, PMax, Demand Gen, and pretty reporting, but still be immature if it is optimizing for the wrong outcome.
2. Search Operating Model.
Weight: 20 points.
The present, riding motorcycles, is about smart bidding, broad match, responsive search ads, and your coverage depth at 100% for AI Max! It is giving up the need to segment by match type, manual bid adjustments (god no!), and human text customization. It includes automated search term matching, final url expansion, and some new controls like brand controls (use them!) and location-of-interest.
> > How to score Capability?
0: Manual CPC, heavy device/hour/day bid modifiers, exact-match obsession, SKAG-era structure.
1: Smart Bidding exists, but you re excessively slicing campaigns by match type, device, or tiny keyword themes.
2: Responsive search ads and some Smart Bidding are live, but the account still reflects a control-first mindset.
3: Core non-brand search uses Smart Bidding with broader matching logic, and, important, RSA-led creative.
4: The team is on max AI Max, and uses controls selectively for governance, not for micromanagement.
Score from 0 to 4, see legend above.
Horse rider tell: “We still believe performance comes from sculpting the account harder.”
Motorcycle rider tell: “We’ve simplified the structure and given the system room to learn. We use controls sparingly, primarily to protect the brand.”
> > How to score Depth?
Truth Question: What percent of our non-brand search budget is still being run with an AdWords-era control mindset?
The search operating model is critical because the most desirable biz outcome: SCALE!
Evaluate depth using share of non-brand search spend.
% of non-brand search spend on Smart Bidding.
% of non-brand search using broad / modern matching approach.
% of eligible search campaigns using AI Max.
To simplify, you can use AI Max row, because it covers broad customer intent matching (beyond KWs), creative optimization, and final URL expansion.
0 to 4, see legend above.
> > Actions to arrive at the present?
Based on your score above… To win, my model will identify your personalized actions from:
- Migrate key campaigns to Smart Bidding.
- Stop relying on manual bid modifiers for performance steering.
- Broaden match philosophy on non-brand.
- Test AI Max where available rather than assuming old keyword logic is still the ceiling.
At the end of this stage, we erase the fantasy that humans can out-manage auction-by-auction intent better than Google’s AI.
The Journey Continues!
The Google Ads maturity model contains four additional dimensions:
3. 1P Data & Audience Intelligence.
4. Surface Breadth & Campaign Mix.
5. Creative & Landing Page Adaptability.
6. Operating Cadence & Governance.
These dimensions help you out-smart your competition – while your competition tries out-spend you!
TMAI Premium subscribers can simply email me if they can’t find TMAI #508, or would like the automated Excel model to identify if you are a Legacy AdWords Operator (score: 0 – 29) or a Modern Google Ads Advertiser (70 – 84), or… are you the unicorn at 85 – 100!
Based on your scores, the model will provide personalized recommendations for action.Tips to Avoid Self-Deception Scoring.
This is not an exercise to fake prove to your CMO that you are an 86. Fooling is not progress. Do a real assessment, even if you keep the scores to yourself.
To ensure zero self-deception I recommend:
A. Score Depth before Capability. It ensures your ego won’t hijack the room.
B. Audit the top 80% of your Google spend. Don’t let edge cases and pilots distort the truth.
The truth will set you free. 😍
Bottom line.
The old game was: Can the human out-manage the account?
The new game is: Can the company feed the machine better truth, better creative assets, better customer value signals, and give it enough room to learn?
The cultural pain of learning the new game is worth it because, depending on your current maturity score…. 3x, 5x, 20x Revenue and Profits await.
Not to mention how much more fun it is to give up the soul-sucking work of the AdWords world. Scores above 69 make work meaningful, even joyous.
Carpe diem.
The post AI Ready? Google Ads Maturity Model. appeared first on Occam's Razor by Avinash Kaushik.
A couple of years ago, I was doing a Strategic Consulting engagement for a global company that operates in 75 countries. The scope was to build out a Marketing strategy for the next generation of success. I was immensely grateful for this fun and deeply challenging opportunity.
In an early meeting with a sub team, they shared that the primary success of their Marketing campaign was the metric Cost Per Session.
I’d never heard of it. In. My. Life.
And, I had a couple of decades of experience. I’d worked with the largest companies on the planet. I’d authored two bestselling books on Analytics, in multiple languages. I’d helped invent entirely new Analytics tools!!
The only CPS I knew was Cost Per Sale.
Now, there are plenty of poor metrics. Take Impressions & Views. They are so value deficient, I would call them “things” and not metrics.
[TMAI Premium Subscribers: Please review the invaluable guidance in TMAI #459, #460: Impressions Suck! If you can’t find them, please email me.]
Still, Cost Per Session surprised me by its existence because I could not believe anyone would consider that as the end point of what their job was in Marketing. Just shovel traffic, and do it as cheaply as possible?
☹️
Pause.
Deep breath.
I’m going to come back to Cost Per Session. If you’re in a gun to your head type situation, I’ll share what you can measure instead to suck less. Even more valuable, I’ll share why as Google heads to AI Mode, a focus on Cost Per Session will harm your company exponentially more.
This post was first published as TMAI Premium #463. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff — built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.Outcomes Over Activity.
My objective is to protect the CMO from the CFO. As in, ensure that the client is executing a marketing strategy aligned with the awesome vision of the CMO, AND that all that activity is producing results that can withstand a CFO’s strict scrutiny.
This turns out to be a bit of a pain for the CMO, as it forces a bit more discipline than they might prefer and a bit more accountability on the marketing organization than they prefer.
After a period of adaptation, this pays off handsomely by identifying the incremental business impact of marketing to the CFO. Quickly followed by ever-increasing marketing budgets.
A real-life case study of that shift in emphasis.
The most common reports floating around your company likely look something like this:
Google Advantage+, OMG, YES! Way to go!! Na na na na Email. Sucks to be you!!Sorry. What I mean is Google Advantage+ does a better job of eliciting a higher response rate.
We celebrate this.
We ship more budget to Mountain Park.
If you have an engaged CFO, or legacy-minded CMO, she/he would ask what the outcomes were from the activity above.
No biggie, we add our normal favorites like Revenue and Conversion Rate and share:
Google Advantage+ delivers.More Orders. More Revenue.
Joy!
When you are assessing the impact of your Owned, Earned, and Paid marketing efforts, at the very minimum, go all the way to Outcomes.
I know that sometimes this is hard because you are a B2B company, or you are a B2C company with a longer sales cycle, or you are a pharma company where the Outcome is a doctor writing a prescription.
I still encourage you to go to Outcomes. In all of these cases you can either track the direct site Outcome or you can measure a Micro-Conversion and times it by an Average Lead to Offline Conversion Rate and Average Outcome Value and get a working set of numbers. These might only be 85% accurate, but they are a heck of a lot better than just looking at the Activity!
Now… If you have a CFO who truly cares about Marketing, they will want to fund Marketing to the max. But, they have to look across all company opportunities (Stores, Support, Product, Eng etc.).
Hence, she will ask you for one more thing.
Accountability Over Outcomes.
All of that Marketing was not free.
Google Advantage+ costs you money. Email costs you money.
Start by collecting those inputs. I recommend Campaign Cost and Cost of Goods Sold (what it cost you to make the products your marketing sold).
Here’s that lovely picture.
It is not surprising that Google Advantage+ costs more.We want to understand COGS – you can do actual or apply an average percentage across products, none of this will be reported to the SEC and hence good enough is good enough.
Once you have the two inputs into Marketing, you are ready for the Accountability view.
$17k in Revenue minus $7k in Campaign Costs minus the Cost of Goods Sold gives you a Profit from Google Advantage+ AI-powered campaigns of $5k.
You can choose the level of accountability you want to demonstrate to your Marketing Loving CFO.
If you want less CFO love/budget, you can use Return on Ad Spend (ROAS).
[Note: TMAI Premium Members, review TMAI #455: Minimum Acceptable ROAS? 8. It will change your Performance Marketing strategy forever.]
The tables now flip. Google A+ at 2.4 looks significantly worse than Email at 9.6 ROAS.
A different view from both Activity AND Outcomes.
Depending on your company culture: Earning you instant gratitude and thanks from the Paid Media team OR the Paid Media team becoming defensive and…
ROAS gives Marketing full credit for Revenue by not accounting for Campaign Cost (ad spend). Hence, it vastly inflates Marketing’s impact. A smart CFO will see through this.My recommendation: At least aim for CFO like (if not love).
You can do that by computing Return on Investment (ROI). It subtracts from the Marketing’s claimed Revenue the Campaign Cost.
The tables continue to flip. Google A+ has a big percentage drop to 1.4 and Email has a smaller percentage drop to 8.6.
Don’t stop the accountability train. Aim for CFO love!
Compute Profit on Ad Spend (POAS). You can see the formula above.
With a POAS of of 1.7 Google A+ is significantly worse than Email.
You are not going to believe it; this is not the end of the story.
That comes by holding Marketing to account for the budget, and calculating Profit On Investment (POI).
Time to cry.
For every $1 you are sending to Google, your campaign is sending back $0.70 in Profit. :( If this was your data, you exist to provide employment to Google’s Advertising Sales team.
Email delivers a profit of $5.7 for every $1 in spend.
Does it matter that your Revenue from Google A+ was 12x higher than Email?
This is why every CMO who wants to grow Marketing budgets YOY, every CFO who truly loves Marketing demands their team measure Accountability over Activity.
Here’s the complete picture:
You’ll use the above to make decisions.You’d expect your Agency to use the above picture, and more (!), to make decisions on your behalf to drive ever higher POI.
Your VPs of Global Marketing, your CFO, receives something much simpler:
If your company culture is against calculating POI, my recommended Priority 1 (P1), you can use POAS (P2).If the culture rebels against Profit or there is simply no way you can calculate it, go one step higher to ROI (P3).
Don’t lower your standards and measure ROAS (but, if you do, the formula is above).
Action: Should You Cut Google Advantage+ Budget?
Yes.
The results clearly illuminate that the way you are spending money on Google (or Meta) you need to immediately stop. In fact, with such poor POI, you should have never started.
Yet.
Recognize that Email only brought you 14 Orders, vs. 173 from Google Advantage+. Sure. You lost a lot of Profit on every one of those 173 orders. But there is the promise of scale.
Google and Meta are successful businesses. They are built primarily on Advertising Sales. That means countless companies are getting high green POI. In this instance, chances are the problem is you, and not the platform.
My recommendation:
Step 1: Cut your Google/Meta spend to zero immediately. Send a message, it is not ok to be the giant sucking sound on company Profit. Pause a week, a month. Let the message sink in. It won’t be easy, there will be massive alarm bells that TRAFFIC IS DROPPING, REVENUE IS DISAPPEARING. Remind everyone, Profit is having a positive recovery at the same time.
Step 2: Offer your internal Paid Media team, your external Agency (if you have one), and your Meta/Google Sales team (if you have one), an opportunity to deliver green POI.
Step 3: Ideally, that invitation kicks off a fresh three-part strategy: A. What intent is available on the ad platform? B. What new set of tactics need to be activated to match that intent with audience, creative, and offers (if relevant). C. Mr. Zuckerberg has spoken of completely automating all Advertising, just send cash & one image per product. Great. What AI-Powered features are we actively using to turbocharge tactics (B) to engage with optimal intent (A).
Step 4: Simple A, B, C strategy, of course, needs lots of smart work underneath it to unlock the scale once more – this time, making tons of Profit vs. currently actively eliminating Profit.
Step 5: Keep spending on Google, Meta, Tiktok, Snap, WeChat, until you see high green POI.
[How high is high? That’s the job of your CFO to identify. Oh, and also what’s too low – something clearly missed in the case study above.]
Cost Per Session | Bye, Bye.
At this point, you recognize the utter futility of letting any Marketing team use Cost Per Sale as a Success KPI or a Metric or, dare I say, even an Influencing Variable.
Right?
[Note: Premium members please see: TMAI #448: KPIs, Metrics, Influencing Variables.]
The Cost Per Session of Google Advantage+ was $14.
Stopping your success measurement at the Cost Per Sale KPI might be delivering job promotions to Marketers, ever higher fees/rewards to your Agency, with the Ad Platform laughing all the way to the bank. You saw the POI of a $14 Cost Per Sale above.
If company culture, leadership issues, or a mass hypnosis prevents you from stopping this unprofitable behavior, my advice is to suck less.
Instead of using Cost Per Session, shift to measuring Cost Per Non-Bounced Session.
You received 510 Sessions from Google Advantage+.
A spend of $7,200 translates Cost Per Session: $14.
The Bounce Rate was 52%. 245 Sessions were: I came, I puked, I left (my definition of Bounce Rate). Completely unproductive (especially since you Paid for each of these people to come!). Take them out.
Cost Per Non-Bounced Session: $27.
With these new, more reality reflecting numbers, there is a higher potential that your internal team and your external Agency (if they are aligned with your interests) will look at $27 and say wait, that sounds crazy high for just a Session, maybe we should dig in further and revisit our tactics.
Perfect.
Sucking less.
There is one more reason you should dramatically deprioritize Cost Per Session focus: SEO implications in an AI Search world.
Google recently shared this collection of guidance for how to do SEO for AI Search. Specifically for AI Mode – a ChatGPT or Perplexity type experience from Google. You should read and start to activate Google’s guidance immediately.
[Note: Premium Subscribers, it is crucial to understand the underlying changes to Search – for both Paid and Organic. Please review: TMAI #412, #413: AI Search: The Sky Is Falling!?]
Google’s guidance for better AI SEO has this important, relevant bit:
Understand the full value of your visits
We’ve seen that when people click to a website from search results pages with AI Overviews, these clicks are higher quality, where users are more likely to spend more time on the site. Why is this? Our AI results may give people more context about a topic overall, and display more relevant supporting links, than with classic Search. This may provide a more engaged audience and new opportunities with visitors, but you might not optimize for these if you focus too much on clicks instead of the overall value of your visits from Search. Consider looking at various indicators of conversion on your site, be it sales, signups, a more engaged audience, or information lookups about your business.
Google’s saying: Don’t use Cost Per Session. Don’t incentivize Cost Per Session. Don’t focus on one-night stands.
If I did not persuade you to disincentivize Cost Per Session, I hope Google did.
Bottom line.
If you want to support your CMO’s career – because that is also very good for you -, prioritize a focus on Outcomes over Activity.
If you want to really do that, you need to ensure Marketing’s impact is resilient and incremental, prioritize a focus on Accountability over Outcomes over Activity.
It is not lost on me that this is difficult, it demands a ton of smart thinking, a lot of hard work, some uncomfortable conversations. In exchange… You get an AI-disruption-proof career.
Carpe diem.
The post Smart KPIs: Accountability Over Outcomes Over Activity. appeared first on Occam's Razor by Avinash Kaushik.
Last year, nearly all of the top 50 telecasts in the US were connected to sports.
The 97th Oscars, SNL 50th, a 60 mins interview, an episode of The Floor and 67th Grammys were the exception.
27 was “NFL Weather Delay,” turns out we will watch anything with sports in the title. 😊
Sport is the last bastion of people watching live TV, and together. That is why streaming giants are getting into acquiring sports telecast rights.
It is unsurprising then that marketers have been rushing into all sorts of sports advertising. Sponsorships of teams, dresses, balls/bats/cars/horses/sideboards/back panels for interviews. Running expensive paid media ads. Naming stadiums. And, more.
At some point, a smart Sr. company leader asks:
Look, I’ve enjoyed the hospitality suite, I’ve loved shaking hands with Tom Brady, and my mother-in-law thinks I’m da bomb for getting her into the clubhouse at Augusta National. But for $38 million dollars in marketing, is it doing anything for the business?
Darn. Consequences. 😊
Let’s answer that question today. How to measure the impact of “Sports Marketing.”
This post was first published as TMAI Premium #497. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff — built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.Prologue.
It is critical to appreciate that all laws of marketing apply to sports marketing.
If your paid media creative and “story” gets lost in the 1.2 seconds of attention on TikTok, the same happens to your logo on the sports car/t-shirt/back panel etc. In fact, in those situations since the purpose of watching is the game, you get even less attention.
(Test: Super Bowl champion Seahawks have only one corporate logo on their uniform, and have had it for decades. Do you know which one? It’s like the most obvious slam dunk answer, yet I bet… you don’t.)
If spike and silence is the kiss of death for your normal marketing, it is exponentially worse for sports – getting the halo from the sport / player being transferred to your brand takes 5x more spike and sustain (because, delicious irony, the sport is getting in the way).
If frequency of 3/User/Week (NOT “avg freq”) is the sweet spot for normal marketing, with sports the “organic placement” is going to get ignored. You still need to get a frequency of 3/U/Wk to persuade for xy weeks. It will come from sports marketing with good old paid media (with the player and sport taking over your creative).
If an occasional normal marketing post by your “massive global” influenceris a (colossal?) waste of money as it gets barely any reach via Organic Social, expect the same to occur with your sports stars carrying your water bottle on to the court or making an occasional post about their sponsor Atlassian.
[Note: TMAI Premium subscribers, please see how to truly win Influencer/Organic Social Marketing: “TMAI #479: Organic Social: Operational Playbook for Winning.” It will save you a ton of money and stress. If you can’t find it, just email me.]
If launching small pop-ups and boutique events has little business impact in normal marketing, beyond the few attendees, your hospitality tent at the Kentucky Derby or luxury suite at the SF Giants stadium will make your CXO and attending clients happy, those few clients might buy more/renew the SAAS contract, but there is little scale.
(Though, I cannot thank John enough for the many invitations to the SF Giants suite! I am open to additional invites. 😊)
If you follow my advice re optimal measurement below, you’ll discover this “dirty little secret” of sports marketing:
- Budgets allocated to sports marketing are best thought of as budgets allocated to developing creative.
- When sports marketing delivers business impact, it will all come from Paid Media Amplification of that creative – over years (with 5 being the minimum). Because many, many, many brands struggle with creative and storytelling.
-
Creative is responsible for 60% to 70% of all brand marketing success.
It can be super profitable to hand over your creative and storytelling to sports, inherit your brand values from the sport/athlete.
Make this decision consciously. Understand the implications.
Then.. MAKE SURE to ask for a ton of money to put paid media behind this creative – and the MMM will prove incremental profitability. I guarantee it.
[Note: Premium subscribers, please activate the groundbreaking Brand Marketing framework in TMAI #496. It will change your professional compensation trajectory.]
Higher Order Bits.
My sports marketing measurement playbook consists of five distinct levels of sophistication: L5, pre-basic, to L1, gold standard.
My “Impact Intelligence Score” (IIS) will rate how smart/informed the company is about business impact at each level. Scale: 1 to 10.
I’m going to use a made-up company, CloudPixel sponsoring the San Jose Earthquakes (my local soccer team).
Level 5: The Opening Gambit: Vanity.
Speed of Impact: Superfast.
Every brand starts here. Sadly, far too many stay here.
L5 is the land of Big Numbers. They look very impressive on their slide decks (“450m IMPRESSIONS”). At this level, you are measuring activity, not outcomes.
L5.1. Social Media Engagement.
You’ll measure the best social media metrics: Conversation Rate. Applause Rate. Amplification Rate. Reported by each social platform. For your company’s organic posts or the sports team/athlete’s organic social posts. Bonus points for measuring sentiment in conversations.
Unless you are doing something wrong, this activity should happen immediately, you can measure it just as fast, and often all the data is free.
Tip: It is a good idea to establish measurement windows (0-24hrs, +72h, +7d) to start to get a sense of decay rates. Establish baselines, and see how the future is against that.
L5.2. Sponsorship Recall.
Nearly all decent sized sports sponsorships will come with my nemesis, the “Ad Recall” metric.
Post-exposure surveys typically ask fans which brands they remember as sponsors – hopefully they say CloudPixel. Surveys could be at stadium exits, online on community sites, or via the Earthquakes CRM/newsletter sample.
Tip: Often these surveys use an aided approach, which lowers signal quality. Are you aware the CloudPixel is a sponsor? Ask for unaided measurement. Split by “exposure intensity” (ex: watched full match, highlights only), and identify deltas.
L5.3. Earned Media Value. EMV.
Also called: Ad Value Equivalency. AVE.
The biggest fav of agencies selling you sports marketing packages. The biggest “con.”
Software counts every appearance of the CloudPixel logo/name anywhere, for any number of seconds. That “placement” gets multiplied by the most expensive ad you can buy. You get told: Hey, all those placements would have cost you $72 million if you bought ads.
EMV assumes all exposure is good exposure, ignores the quality and relevance of the audience, the lack of a dedicated marketing message, and the hyper-cluttered environment. Hence, EMV is widely discredited by major marketing associations (and all causal marketing measurements).
Tip: Stick to one vendor for consistency (rotten apples to rotten apples is sincerely a sound methodology), and, obviously, track trends over time and report deltas.
[Note: Premium subscribers, please also see: TMAI #446: Brand Love Is Not A Helpful KPI. All three of the above will be reported to execs as “brand love.”]
Level 5 Impact Intelligence Score: 1/10.
No causality, no financial linkage, low signal quality, low decision readiness.
We measure inputs, activity, are we visible, exposure, yada yada yada. We are literally measuring the noise. There is little to indicate if it is good noise, if there is any value.
Certainly, measure it in the first few weeks of the campaign. In week five, move to L4.
Level 4: Quest for Brand Impact.
Speed of Impact: Months.
First step toward understanding human impact, via a major upgrade… True test vs. control measurement of brand impact! No pre-post!
For our CMO: Did sponsorship shift what people think and feel about CloudPixel?
[Premium Sub Reminder: Must. Review. TMAI #496. For think and feel definitions.]
L4.1. UBA Lift.
Unaided Brand Awareness is a challenging metric to move. It is a long-term revenue positivity driver. An effective sports marketing program should be able to move it. Even if it takes time to move.
[Never ever never do short-term sports marketing deals, that is flushing money down your you know what because of the long lag to even limited business impact.]
L4.2. Consideration & (Purchase) Intent Lift.
When you report this, your CFO will lean in!
You are in the top three brands, next time I’m in the market for SASS software.
You are my #1 choice, next time I make a purchase.
Powerful, no? Hard to move. Deliciously proven medium-term revenue impact.
The methodology to measure L4.1 and L4.2 are the same: Our traditional brand tracker type survey instruments (OTS, Panels, etc.). Exposed consists of people who demonstrably saw our sponsored content – either the broadcast itself, highlights, our paid ads amplifying the sports creative, etc. Control will be similar people who were ideally not exposed or only minimally exposed.
Relo Metrics, Nielsen, Kantar, are among the vendors we can use.
Tip: Use propensity score matching to ensure that the test and control groups are truly comparable. Instrument variables to account for external factors (like other company activity – from other advertising to product launches to competitor activity). Six to nine months in, start to segment by audience type (gen pop, soccer fan, decision maker, etc.).
Leven 4 Impact Intelligence Score: 4/10.
Causality some, financial linkage low, signal quality medium, decision readiness, low.
Whie self-reported intent ≠ actual purchase behavior, still there is a ton of science in measuring perception shifts, and the causal impact specifically of sports marketing.
[Premium Subscribers please see TMAI #426, for an excellent brand metrics deep dive.]
L4 measurement is even more helpful for B2B companies, with extremely long or complex sales cycles.
Start measuring in the first quarter of the sports marketing program and then keep a periodic pulse going for test of the (min) five years of investment. In a handful of quarters, you’ll be ready to move to L3.
The Journey Continues.
Achieving level 4 will get you into the game, but to truly get to proving to your CFO, your Board, that your Sports Marketing investment is delivering a profitable impact… There are three more levels to nirvana…
Level 3, “The Heart & Mind Influence”, gets to insights into compounding brand advantage.
Level 2, “The First Digital Trace”, helps you identify to financial outcomes driven.
Level 1, “Show Me THE MONEY!,” ensures you prove the ultimate: Incrementality!
L3, L2, L1, are covered in detail in TMAI #498. If you are a Premium Subscriber and can’t find that edition, or a new Premium Subscriber, please email me for a copy.
Bottom line.
Sports marketing can be immensely effective – even transformative. Ex: Rolex and tennis.
With Level 5 measurement, you can tune creative and activation to ensure your $$$ are spent on getting you visibility.
With Level 4, you get to prove to the CMO that her choice to invest in sports marketing is proving initial hints of something actually valuable: foundational shifts in the Big 3 Brand KPIs.
Stop sponsoring in the dark. Turn the lights on.
PS: Special Bonus: A podcast version of the Sports Marketing Playbook!
The post Sports Marketing Measurement Playbook. appeared first on Occam's Razor by Avinash Kaushik.
HiPPOs still hand down most decisions in a company – even when surrounded by piles and piles of reports with metrics galore.
Marketers still use the “funnel” to imagine and allocate budget, people, actions – even when there is multi-decade data that the funnel is a lie.
Company after company misses trends and goes kaput – even after investing in multi-million dollar data projects to build clean rooms, unified consumer view cloud-based business intelligence platforms.
☹️
The core challenge isn’t data scarcity; it is insights latency.
Which impacts your ability to follow my advice to deliver IAbI, and not data.
Here’s the traditional analytics workflow being practiced in your company:
1. Report Generation. Hours and hours of standard reports/dashboards.
2. Manual Analysis. Hopefully, laborious segmentation of known knowns missing subtle non-intuitive patterns. AKA High-dimensionality data challenge.
3. Insight Extraction. Identifying the most important findings, extracting context from Marketers, Finance, Sr. Leaders, creating presentations.
4. Exec Last-Mile Barrier. Data competing with other priorities, insights missed, misinterpreted, translation to action challenging – to say the least.
A process that is fundamentally reactive and linear. It struggles with tracking hundreds of variables per user engagement, non-linear patterns (path analysis anyone?). Humans are ill-suited to find a needle in the haystack – identifying truly significant anomalies or emerging trends within massive datasets.
☹️ ☹️
It does not have to be this way. anymore.
It is time to hand control of Marketing Analytics over to AI! Online, offline, digital, everything.
😊
AI can act as a force multiplier and overcome the above limitations.
A. Pattern Recognition at Scale. Machine Learning (ML) algorithms are awesome at finding complex, non-linear relationships and hidden clusters within massive high-dimensionality datasets.
B. Automating the Mundane. As I’m sure you’ve seen in your use of ChatGPT, Qwen, others, AI can automatically generate insights from routine data, it can flag anomalies instantly, and surface the most statistically significant changes.
C. Predictive Power. Rather than reactive, what happened, AI is exceptional at what’s likely to happen, thus solving the insights latency. It is worth noting it can personalize this at a massive scale – dynamic segmentation, tailored experiences, value something impossible with manual rules. AND, do this uniquely for the needs for every human in your org!
D. Continuous Learning. AI’s real superpower. AI models adapt as new data flows, they constantly refine their understanding of user behavior and system performance – at a massive scale. (It would be equivalent to the Analyst earning a Bachelor’s degree in a new field every few weeks!)
Handing control of Digital Analytics over to AI achieves this profound shift: From Analyst-as-reporter to Analyst-as-strategist. From data puking and insights hunting to validation and activating action.
😊 😊 😊
It is time to rebuild analytics from the ground up.
You’ll remember I originated the 10/90 rule of Analytics 20+ years ago.
“If you have $100 to invest in smart decisions, invest $10 in tools and implementation, invest $90 in humans who will analyze the data!”
Here’s my new 10/90 rule for success via investment in Analytics:
“If you have $100 to invest in smart decisions, invest $10 in brilliant human analytical strategists, invest $90 in AI activation.”
In fact, over time the $100 is likely to reduce to $80, then $70, and maybe less… While the quality of decisions, the scale of intelligence and automation, will exponentially increase.
Incredible, no?
Let’s learn how to activate this immense value.
This post was first published as TMAI Premium #492. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff — built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.Activating AI Power.
AI is not yet AGI (Artificial General Intelligence), and certainly not SGI (Super General Intelligence).
[Note: Premium subscribers dive into these key concepts in TMAI #457, learn how to apply them across all business functions. If you can’t locate it, please send me an email.]
Today, activating the awesomeness above will take human grit, intelligence, and persistence. Things won’t be perfect.
Your True North: Somewhat failing to activate my recommendations is 25x better than your present. And, as a bonus, you’ll be ready for AGI.
I have ten specific implementation ideas for you to turn your digital analytics over to AI. I hope they’ll spark a dozen more in your team.
- Predictive Analytics via Propensity Modeling.
Impact Potential: Transformational.
Human-powered digital analytics tells us who converted. AI can tell us who will convert!
There are thousands to tens of thousands of humans on your site, using your apps today. Instead of spreading your budget, attention on all of them, you can focus on high-propensity humans.
ML algorithms thrive on pattern recognition across hundreds of variables, and thus identify subtle combinations of behavior that signal conversion readiness (or whatever your digital objective is). IMPORTANT: Unlike rule-based systems (if user views pricing page three times, tag as hot or if a user has seen handbag 1, 2, 7, give them a discount), AI models consider non-linear relationships and interaction effects between dozens/hundreds of variables for a more brilliant understanding of human intent and what will happen next.
Framed simply: What is the exact probability THIS human will convert/upgrade/churn in the next N days?
AI Approaches and Algorithms to explore, stress test, and embrace:
- Gradient Boosted Machines (XGBoost, LightGBM). Currently, the gold standard for tabular data prediction. These algorithms excel at conversion prediction by combining many weak predictive models into a highly accurate ensemble.
- Random Forests. I have loved using RF when I have a need to understand feature importance. Ex: Which behaviors most strongly predict conversion?
- Neural Networks. The grandpa of AI. For massive datasets with complex, nonlinear relationships, deep learning architectures can uncover patterns other models miss.
- Survival Analysis. Good old statistics. Predicts not just if, but when a user will convert, enabling perfectly timed interventions. Each business is unique; you might use a couple from above, just one, or all of them to solve different propensity modeling opportunities (emails, B2B conversions, internal HR use for people who are likely to quit, etc.). One of them you should have activated in the next six months.
Practical Example.
From my experience: A propensity model using approx. 90 behavioral features (scroll depth, product views, cart additions, days & visits in experience, etc.). The model scored each user in real-time, allowing the ecom COE to:
- Serve dynamic offers to high propensity users.
- Adjust bid strategies for retargeting ads based on conversion probability.
- Identify “at risk” users who showed high intent but did not convert (so we proactively intervene vs. lose to competition). Potential Outcomes For You.
Looking across my work on three continents, focusing on ecommerce:
A. 35% – 60% improvement in Conversion Rates for the targeted segments.
B. 20% – 35% reduction in acquisition costs due to more efficient ad spend.
C. Not easily quantified qualitative impact of shifting from reactive to proactive marketing.
Over the last three years, Propensity Modeling has been my most monetized, highest now-potential, game-changing action in handing over digital analytics to AI. Every quarter you don’t activate it, you are falling two to three quarters behind.
[Note: TMAI Premium subscribers will recall three editions dedicated to sharing a roadmap for building AI-powered Propensity Models. TMAI 378, 379, 380. Email me if you can’t find them.]
2 Advanced Customer Segmentation.
Impact potential: High.
You should not be surprised that this is so important. My blog was born May 2006; this is from then: Excellent Analytics Tip#2: Segment Absolutely Everything.
Most analytics teams segment users by demographics or broad behavioral categories (e.g., “mobile users,” “TikTok ad visits,” “logged in”). These segments are often too broad, and miss thousands of nuanced behavioral patterns. Creating more relevant, precise, sophisticated segments manually is extremely time-consuming and limited by human bias, human knowledge (little awareness of known unknowns, and none of the unknown unknowns).
Unsupervised learning algorithms specialize in finding natural clusters in data without predefined categories. They can process dozens of behavioral dimensions simultaneously to identify segments that are statistically distinct rather than intuitively appealing. They can get to the unknown unknowns – hidden well below the human capability surfaces.
AI Approaches and Algorithms to explore, stress test, and embrace:
(This is less AI, a lot more algorithms and old school ML.)
- K-means Clustering. Thousand-year-old workhorse algorithm for segmentation and grouping users based on behavioral similarity across multiple dimensions.
- DBSCAN. Full, cute, name: Density-based spatial clustering for applications with noise. Instant love, no? 😊 Specifically awesome for identifying outlier segments or detecting novel user behavior patterns. (If your site gets more than 100k visits/day, there are thousands of novel user behavior patterns in there!)
- Gaussian Mixture Models. Few models handle ambiguity better than hard clustering approaches, when segments overlap probabilistically.
- Hierarchical Clustering. Fifteen times a day, Analysts have to drill from broad categories to highly specific micro-segments. Segmentation trees created by hierarchical clustering are a perfect solution.[Note: Premium newsletter subscribers for a practical application of using decision trees, please see TMAI #283. It shares how it helped 10x the impact of YouTube video campaigns with insights hidden in data (from Analysts!).] There is immense untapped potential to be extracted by applying unimaginable scale to segmentation via AI and algorithms.
Practical Example.
Tying this back to an example from my 2006 blog post, but applied recently to a B2B (SaaS) client. We applied clustering to session data across 28 behavioral dimensions. Instead of the standard free trial users segment, the algorithms identified:
- Segment A: Feature explorers, who try many features quickly.
- Segment B: Cautious adopters, who read documentation before acting.
- Segment C: Social validators, who always check testimonials first.
- Segment D: Price-sensitive evaluators, who immediately navigate to pricing. Tying these segments to outcomes, working backwards to influence them, allowed the team to customize onboarding of the trial experience in real time, the content and flow of the product, and, obviously, subsequent messaging to dramatically improve activation rates!
Potential Outcomes For You.
Reflecting on my clients and work:
A. 25% – 50% improvement in Conversion Rates from behavioral targeting based on algorithmic segments.
B. 60% to 75% reduction in analysis time dedicated to customer segmentation.
C. Not easily quantified qualitative impact on customer joy and company revenue from a deeper understanding of potential customers and their behavior.
Big picture: What took weeks of manual cohort analysis (assuming you could even guess them all right), now happens automatically (with dynamic segmentation, dynamically updated, at unimaginable scale and with improved precision).
- Voice-of-Customer Integration with Behavioral Analytics.
Impact Potential: High.
Another one of my old web analytics dreams has come true. Early readers will remember my, at the time, revolutionary Trinity model for Analytics. 19-years later, I can AI it!
Survey responses, support tickets, chat transcripts, call center voice recordings, and social media mentions live in separate systems from behavioral analytics. Analysts struggle to connect the why with the what (failing Trinity). This leads to incomplete understanding of user motivations, frustrations, and unmet needs. Solving this at scale with humans is futile.
One of the key leaps of modern AI is multi-modality – the ability to understand text, images, voice, and video at unimaginable scale and incredible precision.
Multimodal AI systems can process both structured behavioral data and unstructured text/voice data simultaneously. Advanced embedding techniques allow algorithms to find connections between language patterns and behavioral patterns at scale. Sentiment analysis has evolved beyond simple (and lame!) positive/negative classification to detect specific emotions, urgency, and intent that, a blessing for us, correlate with behavioral outcomes.
AI Approaches and Algorithms to explore, stress test, and embrace:
- Multimodal Transformers. Great for processing text and behavioral data in a unified model architecture, with cohesive, understandable outputs – at scale.
- Cross-modal Retrieval. Helpful in finding behavioral sequences that correspond to specific feedback themes, assessing their quantitative relationship.
- Advanced Sentiment Analysis. Taking our positive/negative past approach to significantly higher accuracy and detailed why patterns by detecting frustration, confusion, excitement, and uncertainty. Applications across every part of the business (including annual employee surveys!).
- Topic Modeling with Behavioral Correlation. A data setup problem to overcome, but then at scale drive discovery of which discussion topics correlate with specific actions or drop-offs (which exist, ex, in your mobile and site quant data).
- Emotion-Action Mapping. Above hinted reflective analysis, now you switch the view to predictive analysis. Connect expressed emotions with subsequent behavioral patterns – driving proactive actions by ensuring you don’t lose a logistics provider or an employee quitting or a massive B2B client not renewing their contract. Practical Example.
This one’s from a pal, in Canada. An ecommerce platform integrated the site’s chatbot data with behavioral data and put in place an AI model to analyze it. Discoveries:
- Positive mentions of sustainability in chats correspond with 3.4x higher lifetime value.
- Discovered emerging complaint patterns about a new feature, three months before negative NPS scores.
- Users expressing “size uncertainty” in chats have an 82% higher return rate. (OMG) Potential Outcomes For You.
A. 8 – 12 points improvement in NPS scores.
B. 20 – 25% reduction in fails (cart abandonment, returns, etc.) from real-time interventions put in place.
C. Not easily quantified qualitative impact on product development from 360-degree customer understanding from connecting the why with the what systematically.
[Note: With life experience, Trinity model became an even more sophisticated and modern Edge model for Analytics. Premium subscribers, please see TMAI #224. And then a refinement of it in TMAI #273: The Analytics Flywheel | Invent Once, Scale Infinitely.]
The Profitable AI-Analytics Journey Continues.
In TMAI #493 and #494, I’d shared additional super exciting ideas to deliver transformative profits via AI-Powered Analytics. Additional activations included:
4. Behavior Targeting & Intelligence (BTI).
Impact potential: Transformational.
5. Natural Language Processing (NLP) for Unstructured Data.
Impact potential: High.
6. Anomaly Detection and Automated Insight Generation.
Impact potential: High.
7. Predictive (Whole Company) Customer Lifetime Value Modeling.
Impact potential: Transformational.
8. Real-Time Pricing and Offer Optimization.
Impact potential: High.
9. Intelligent “Liquid” Merchandising.
Impact potential: Medium.
Give all of the above is true today, I predict that the current type Analyst role will cease to exist over the next 18 or so month. In TMAI #495, I laid out a framework that outlines what the Analyst role will be in Jan 2028, and how you need to get ready for it starting now: TMAI #495: Analyst 2028: S.H.I.F.T For Relevance.
If you are a new TMAI Premium member, please email me for the series above. If you are not, grab an annual Premium subscription here.
Bottom line.
The integration of AI into Analytics represents the most significant shift in our field since its birth as a science.
The organizations that will thrive in the coming years aren’t those with the most data, most Analysts, most spending on Analytics. They will be the ones who can extract the most insight from their data with the greatest speed. i.e., reduce insights latency and increase automation.
AI and advanced algorithms provide the tools to make this possible, transforming analytics from a practice of historical reporting to one of predictive intelligence and prescriptive optimization.
Carpe diem.
PS: It is only appropriate that I share with you an AI-generated summary visual of this blog post! For your slides…
The post Bye, Bye Human-Powered Marketing Analytics. appeared first on Occam's Razor by Avinash Kaushik.
There’s a scary Giant hiding in your closet.
It imposes hidden costs that, when accounted for, transform your claim that advertising is adding business profits. Short-term, long-term. The scary Giant changes your OMG! to omg?
There’s an incredible return from investing time and love in identifying your Giant costs. Ex: Identifying non-working media costs, by calculating them for the core and sub-components.
There is only one other thing more important in Advertising (incrementality).
[Note:Newsletter Premium Subscribers: If you can’t locate TMAI #437: Compute Non-Working Media Costs and TMAI #411: Proving Marketing’s Incrementality, just hit reply.]
This got me thinking about how frequently we throw around the key performance indicator (KPI), Return on Investment (ROI) – without being careful how they are computed or transparent about what they include or exclude.
We simply claim: Our Performance Agency is delivering an ROI of 4!
The claim’s implication:
For every $1 our Agency spends on Ads, they are delivering $4 back. HURRAY!!So today… Let’s interrogate that 4. What is it? Can it be trusted to reward your Agency?
A question to answer by the end: Does your Agency impact survive calculating ROI 4?
This blog post was originally published as Premium edition #438 of my newsletter. Weekly, I share actionable insights and hidden patterns to stay at the bleeding edge of Marketing, Analytics, and AI. Sign up for TMAI Premium. 100% revenues are donated to charity.
With the Real ROI, Please Stand Up?
So, what’s ROI?
The most common computation:
ROI = [(Revenue – Media Costs)/(Media Costs)]
Media Costs are typically the Dollars/Renminbi you paid to run ads – on Facebook, Magazines, CTV, Radio, Bing. Sometimes referred to as Advertising Costs.
Revenue is the traceable sum of $$$ earned from running the aforementioned ads.
ROI is often expressed at a campaign level – though you can obviously decompose it by an individual ad, a channel, a group of tactics, and on and on. [Premum Subscribers: This is when the Multi-touch Attribution methodology becomes super important – plese refer to TMAI #434.]
I was reviewing a Client’s QBR for a recent Campaign and sure enough they’d computed ROI:
[Privacy Note: Numbers are real, the visualization is mine. Any mistakes you catch are mine.]ROI = 4!!
[Note: I’m not going to cover the commonly bandied about ROAS – Return on Ad Spend. While a close cousin of ROI, I consider ROAS to be emotionally sketchy.]
The Agency did not know the Campaign’s overall budget, not unusual, as Agencies rarely do (though you should share with them). I’ve added that number to the table above.
An ROI of 4 looks incredible, no?
I offer that the 4 is unreal. It meets the classic definition of fake news.
To sell the shoes / car parts / laptops / eyeglasses / Bluetooth adapters, you had to design them, manufacture them, ship them, store them, and wait for the order to come. Of all those costs, at the very minimum, you cannot ignore the cost to manufacture them.
You sell a pair of eyeglasses for $50, you need to account for the $35 Cost to manufacture them. $35 is known as Cost of Goods Sold (COGS).
Hence, this is a more real news formula of ROI:
ROI 2 = [(Revenue – COGS – Media Costs)/(Media Costs)]OR
ROI 2 = [(Gross Profit – Media Costs)/(Media Costs)]
I call ROI 2: “Gross Profit ROI.”
For the client above, this is a more real ROI the Agency delivered:
For this company, the COGS was 70% of the sale price (expressed as Gross Margin above).After counting that, the amount the company made was $0.9 million, and not $3.2 million. The new, more real, ROI driven by advertising is 0.5.
While heartbreaking, please learn to embrace the 0.5 – or you will never know how to be better.
Wait, wait, there’s more.
Remember, the total budget spent by the Marketing team was $1 million. It is not the $0.6 million being used in both the formulas above.
The delta, $0.4 million, were non-working media costs. [Premium Subscribers: See TMAI Premium #437 for how.]
IMPORTANT: Your Agency spent $0.6 mil, their calculation is right for what they know. You spent $1 mil, it is your job to account for this money.
You must account for the Total Campaign Spend, by using this formula to compute ROI:
ROI 3 = [(Revenue – Non-working Costs – COGS – Campaign Budget)/(Campaign Budget)]OR
ROI 3 = [(Net Profit – Media Costs)/(Campaign Budget)]
This helps us land even closer to the real ROI that your team (not Agency) delivered to the company:
The Net Profit ROI 3?Minus 0.1.
Your advertising campaign lost money.
A shocking realization when you reported ROI as 4 to your CMO. No?
We are not done getting to the business value of this campaign.
There’s one more thing to get to the realest ROI from advertising.
What would have happened if you did not execute this campaign?Would you have lost the entire $3.2 million in Revenue, if you had not spent the $1 million on advertising?Incrementality.
Incrementality!
We who are active practitioners of the art and science of incrementality know that you would have made a bunch of the $3.2 million even if you did not execute the campaign.
I know, I know, it hurts our feelings as Marketers, but sadly, it is reality. Nearly all the sales that come into your company have nothing to do with Marketing!
Let’s do one more computation of ROI, this time accounting for incrementality.
In this case, the Agency did not practice incrementality for this Client, hence, for today, I’m going to assume it is a super high 30%.
What does that number mean?
70% of the Claimed Sales by this campaign, would have occurred any way (store location, product features, seasonality, innovation, reviews on Amazon, whatever else).
Here’s the final, closest to real, formula for ROI:
ROI 4 = [(iRevenue – iNon-working Costs – iCOGS – Campaign Budget)/(Campaign Budget)]OR
ROI 4 = [(Incremental Net Profit – Media Costs)/(Campaign Budget)]
That yields the following Incremental Net Profit ROI (4) results:
We really lost money.The Campaign’s incremental Net Profit ROI (iROI) is -0.7.
A very different picture than the 4 the Agency presented at the start with ROI 1.
Difficult Questions:
What do you and your Agency compute today? ROI 3 at least? Perhaps, ROI 4?Does our journey today explain why the Marketing budget keeps getting cut by the CFO, despite Marketing’s protests that they are delivering 4x return on investment?Special Note | Brand Marketing ROI.
The ROI computations above span a four to six month impact horizon.
For brand marketing campaigns, the impact horizon, will stretch beyond six months.
For such campaigns, we compute short-term ROI #4 using different KPIs (# People Lifted, Cost Per Individual Lifted – both vs. baselines), and different methodologies (true test-control surveys, not pre-post).
And, we will hold Brand Marketing to account for delivering long-term profitability! For that, we will measure long-term ROI #4 with the same KPIs (incremental Profit), but different methodologies (longer impact horizon like advanced attribution modeling, ML-based mix models, and CausalAI).
Radically improving Marketing’s ROI.
Good Marketing can absolutely deliver a magnificent Return on Investment. But how?
For my clients, I take a repeatedly tested in the real world four-step approach to deliver radically better ROI. I did a deep dive into each step, and actions you should take, in Premium edition #440. Here’s the summary:
Step 1. The Marketing Team: Obsess about excessive non-working media costs.Step 2: The Agency: Obsess about highly incremental tactics.
Step 3: The Commerce Team: Why is the Conversion Rate so low?
Step 4: The Engineering Team: Product costs and process innovation
The glorious profit-generating outcome my approach above looks like… You can replicate it in your company…
[Higher resolution: Right Click, Open in a New Tab.]
TMAI Premium subscriber? Please email me for the excel spreadsheet, and the deep dive details of the four step process above.
Bottom line.
Marketing tends to be the first budget to be cut in tough times.
Two reasons:
1. No one at the top of the company quite believes any claim the CMO offers re impact of Marketing (see above).
2. Marketing competes with Engineering, Retail Stores, Customer Service, HR, Factories, Finance for budget – the short-term ROI from all of them is easier to see (and believe).
This is our (Marketing’s) problem to understand, and fix. Here are your standards:
ROI #3 is the minimum standard that’ll survive Board or CFO scrutiny.
ROI #4 will ensure Marketing is among the last budgets to be cut.
Carpe diem!
The post The Best Marketing ROI Formula: Incremental Net Profit ROI! appeared first on Occam's Razor by Avinash Kaushik.
Here’s a sign that you’ve arrived as an Analyst or an Analytics team:
At the first sign of failure reported by the data, most people blame you (Analyst/Data).
Wear it as a badge of honor!
It means your analysis has identified insights that are big enough, important enough, that the recipients get instantly worried.
Ideally, you live in a culture where good analysis identifying poor performance would be warmly welcomed as an opportunity to learn, an opportunity to change, and, for the bravest cultures, an opportunity to change leadership posture (or leaders). What's often a lot more common is to take the easy way out by sowing doubt, undertaking "rationalizations," and/or blaming data (not the performance!).
Let me be emphatic: Scapegoating Data/Analysts is counter-productive. It is a heartbreaking reflection of culture and leadership.
In my, more years than I care to admit, career scapegoating Data/Analysts is a feature of the company’s culture, not a bug. In the rarest of cases when scapegoating data was not a thing, these three nuclear-powerful elements were built into the culture:
A. An understanding that data is usually only 90% accurate. There are always elements we can’t answer or account for. That is ok, the quest for perfection is futile.
B. Let’s use the data we have, and in a blameless spirit, make decisions within confidence intervals, and drive change.
C. If there is a pattern in data that consistently points to a shortfall in results from a leader: Let’s get that leader help to dramatically improve, or, after continued shortfalls, help them transition careers.
A + B + C = Human excellence delivering deeper customer love & outrageous profits.
This wonderful reality is rare.
That hurt's my feelings deep, deep inside – as you would expect for someone who's an Analyst.
Today, let's learn how to recognize Data/Analysts are being scapegoated, and how to wire a culture (and leadership) to ensure that scapegoating is not a defining feature.
This blog post was originally published as editions #305, #306 of my newsletter TMAI Premium. They contain 39 strategies to recognize scapegoating (a selection below), in addition to detailed guidance on models, algorithms, frameworks in the Analyst Fix Thyself section (subheadings shared below). Sign up for sign up for Premium, accelerate your career trajectory!
Common Scapegoating Strategies.
Being able to recognize when data (or the Analyst) is being scapegoated is a superpower. If you can recognize it, you can do something about it.
Here is a selection of strategies deployed when scapegoating Data/Analysts:
We had set Consideration as a KPI, but we were solving for Awareness.
We can’t solve for this metric in the short-term, how can you possibly say the campaign did not work?
You are measuring a “lower-order metric,” we were solving for an “higher-order metric.” (What!)
Sure, these campaigns did not drive any conversions, but why are you not focusing on how many impressions we drove?
How can you possibly say 350 responses are statistically significant, we have 400,000 visits to our website every day?
Why would you show us results from attribution analysis when last-click always made me look good?
Your Test markets are just 6 and your Control markets are 28, that makes no sense for getting accurate results.
Start with the presumption that data is wrong, then proceed.
Last year, one of your analysis was flawed. How can you be so certain this analysis showing failure is believable?
Learn to create win-win situations, even if the data says we have consistently failed for the last two years. Be a team player.
You are not very good at understanding all the context behind the consistent poor performance.
The above is a selection of strategies to question the metrics, question the methodology, and question the Analyst at a personal level. [See TMAI 305 for the complete list – if you are a Premium member and can't find it, just email me.]
Often when leaders are scapegoating your work, or you personally, it is not about the fact that your work is only 95% good enough. The source is an instinct to preserve the status quo, to not look bad to a superior, or to escape accountability for their non-success.
Learn to recognize the 39 strategies used. It is good for your body and soul.
You'll also go on fewer wild goose chases.
Analyst, Heal Thyself First.
When I find myself in situations where my work is being attacked, my first instinct is to look inward.
What can I do better at a personal level?
Are we doing everything we can, as best as we can, to provide intelligent analysis framed constructively (vs. not whitewashed/”massaged”)?
Some interpret this as a blame the victim mentality. I simply see it as approaching the situation with humility.
It is natural for people to jump outward, but coming to it after the inward reflections makes the outcomes multiple times more effective.
I recommend the following four strategies to ensure you have done everything you can to ensure you meet high standards of data analysis and insights generation:
1. Rigorous torture of measurement tools.
2. Consistent practice of data validation.
3. External assessment of your analytics approach.
4. Improving self & team EQ.
See TMAI Premium #306 for detailed write-ups, and links to deep dives.
Now, the entrée and the dessert.
Strategies to Reduce Scapegoating of Analysts.
Let me share seven outward looking strategies to reduce scapegoating of Analysts/Data, “what can we do to improve the system and its incentives.”
1. Pre-identified KPIs.
If you don’t know where you are going, you will get somewhere and then be miserable.
A core strategy for scapegoating Analysts can be undercut by establishing a process, upfront, to decide what the KPI for the campaign is.
Remember, a KPI is a metric directly tied to the business bottom line.
You have an $18 mil campaign focusing on “branding”? Ok. Ok. Cool. What’s the KPI? You’ll get some ambiguous something "brand love" or awareness (ask if aided awareness—which is crap – or unaided awareness—which is great).
You have allocated $8 mil more to Paid Search? Great. What’s the KPI? Clicks? No! Bounce Rate? Nyet. Average Order Value? Metric, not a KPI. Conversion rate? Not without an effectiveness metric. Revenue? Better. Profit? Best.
If you decide upfront which (good) KPI is going to trumpet success, it is harder for any senior leader in Marketing to go after you with a machete after the campaign is done, and results are poor. After all, they decided the destination.
To make this even more solid:
A. Decide what the KPI is before the campaign starts. (I know this sounds obvious, you’ll be surprised how many people move goal posts after the fact.)
B. Iterate identification of the KPI with the finance process that approves the Purchase Order to fund the campaign. This gives you rare strength via all the power that Finance holds in most companies.
2. Pre-identified Targets.
What’s even better than a pre-identified KPI?
A pre-identified KPI with a pre-identified Target!
Of all the strategies I’m recommending, this will be the hardest. There are zero instances of humanity not resisting target setting, gaming target setting, sandbagging targets, and on and on. It does not matter if you are in Marketing or Sales or Finance or HR.
Yet, setting upfront targets will protect the Analyst. Because, the business is declaring upfront what success or failure will look like. If the performance looks bad after the fact, it is not the Analyst saying it is bad or good – the leader/Finance approved the targets. Upfront.
$18 mil on branding to lift Unaided Awareness by 8 points? Awesome, let’s go do it.
$8 mil on Paid Search to achieve $38 mil in Revenue at a Conversion Rate of 7%? Perfect, let’s go put strategies in place to achieve it.
Targets force better behavior by Directors, VPs, CMOs who would have scapegoated you after the fact. Targets will encourage them to look hard at the tactics they are planning to execute, and to do so upfront. This is good for the company.
My recommendation is that while Analysts can help provide input into target setting (to help reduce sandbagging and gaming of the system), targets should ultimately be owned by Finance. An independent third party with a big stick.
Obviously, Targets are also great for Analysts. After the initiative is completed, analyzing which tactics cause results to exceed targets or miss targets can be an invaluable source of learning. They’ll yield smarter out-of-sights.
3. Smart – trusted, but verified – prediction models.
I’ve really, really, leaned into this with teams I help lead. Have statisticians on the team who can help you build smart models to make predictions.
If you can tell the CMO upfront, based on decisions she has made to approve a campaign with xyz goals and abc tactics, that the campaign is going to fail… You remove the entire basis of scapegoating.
You are telling her upfront, before any money has been spent: You are going to fail.
She is not going to like hearing this. You’ve made it almost impossible for her to do what she wanted to do – she dare not move forward with something predicted to fail (and then if it fails … you told her so!).
But. It is a million times better to get this “hate” upfront than after the fact, when the force of scapegoating will be much more intense.
After most campaign details are finalized (audience, budget, duration, KPI, Target, Reach, Frequency, yada, yada, yada), but before the campaign is in-market our team provides the senior most decision maker a prediction:
There is a 32% chance this campaign will deliver 8 points of lift in Unaided Awareness.
It stops everything in the tracks.
Every so often, the VP will say: It does not matter. We are doing this.
At this point, there is a huge shift in accountability: From you to them. This is, as Martha says, a good thing.
Most times, the VP will say: What do we need to do to fix it?
How incredible is this for everyone? So much!
We, the Analytics team, are humble and do trust, but verify exercises on our predictions. [See TMAI Premium #287.] We are correct 87% of the time. There is room for improvement, but 87% is pretty damn good!
Being able to predict, upfront, changes the game. In the best possible way.
Can your Analytics team predict?
4. Lots of in-flight signals of performance.
What’s better than telling people after the fact that they failed or succeeded?
Telling them while they are spending money that they are failing or succeeding!
Doing confident analysis in-flight is hard. How do you figure out what metrics are believable when? What parts of signal quality do you need to worry about? Oh, Nielsen can’t give you TV data for six weeks? Some results take a while to accumulate (offline is a good example, you might not see anything for the first couple weeks, but then it builds and builds—except when nothing happens the whole time!). There are so many challenges. But, if you can overcome them… You have a great anti-scapegoating strategy.
We started with mid-campaign reports (half-way through). Enough time for a signal, and time still left to make changes (because you can tell the executives we predict that if we continue on this path, the campaign / channel / tactic will fail to deliver results).
Then, we got better, and went to 3x per campaign.
Now, we can start with the digital signals, with a measure of confidence, about 7 – 10 days into the campaign. Offline, three weeks into a campaign.
We build a swat team of key folks involved in the campaign, we review these results, we work together to make changes.
Two great benefits:
A. Every change you make in-flight improves the end-of-campaign results (hurray!), and hence reduces scapegoating of analysts.
B. Even if no changes are made, the executives would have heard 3 to 15 times before the campaign ends how it is doing, this reduced scapegoating of analysts.
Win-Win.
5. Presenting the long view.
I’m embarrassed to admit that I’ve only deployed this anti-scapegoating strategy in recent years.
Instead of just focusing on performance in the short-term (a quarter, two quarters, duration of this specific campaign / initiative), show performance of this VP/Director/Region/Product over the last three years or five years.
I call this, imaginatively, the long view.
With a big bang, the long view brings oodles of context with it… For the last quarter’s performance. It shows long-term trends.
It might illuminate that the latest suboptimal performance of the branding campaign or paid search campaign is not an anomaly… Over the last three years, the campaign executed by this team has more often than not failed to deliver against pre-set targets.
It is just as likely to show that the latest bad performance is indeed an anomaly, and let’s lower all the daggers.
When you look at patterns over the long term, you subtly shift the recipients from blaming measurement/analysts.
The challenge with the long view is to ensure it does not become a data puke. Here’s a list to help you start, and keep it to a limited data puke:
In rows: Budget, Audience, Primary KPI, Target, Predicted Success Score, Secondary Metric (Effectiveness, or Efficiency depending on what the Primary KPI is), Marketing Channels.
In columns: Time (month or quarter, as is optimal for you). Facts and performance in each column for that time duration.
Oh, and this long view with patterns is extremely beneficial to analysts as well because it encourages them to cast a wider net for causal factors of good and not-good performance.
6. Building an influential posse.
Don’t go through life alone.
Build a posse.
Analysts can be insular. And, since they have so many problems to deal with, and so many headaches their leaders dump on them, and some can be socially shy, they can stay inside their circle. Some leaders of Analytics teams create an oppressive environment for data. All this can lead to a non-positive, insular, reality.
Isolation is not good.
Every time I’ve built an Analytics team, I’ve worked very hard to build resilient relationships with the Finance, Strategy, and Operations teams. These teams have extraordinary power over what’s planned, what’s done, what’s prioritized, and, yes, what’s measured. You want to have BFFs in all three of these teams.
Only rarely will these teams come to you. You go to them.
Understand what their priorities are, what their concerns are, what their gaps are. Then, reflect on what you have and package up solutions you can deliver to them. Don’t ask for anything in return, help them. Contribute to them being more informed, efficient, and effective. They are humans, they will not forget this.
If they are willing, when they are willing, build your processes (strategies 1 – 5) to include them at key steps. Then, when you are in difficult meetings, you will have others in the meeting who are from different teams be involved in looking at the analysis, and, on occasions when you need it, speak up for you if you are being scapegoated.
Engaging with these teams will also broaden your horizons, help you learn valuable context, and throw up ideas for steps you can be involved with that can deliver better business results.
I don’t mean to limit your posse to the above three teams. Make friends with Procurement and Sales and Customer Service and Product Management, and, of course, Engineering and the VPs at your Agency and your CMO’s administrative business partner (SO important) and… Whoever might be touching the depth and breadth of work that happens in Marketing. Take your time, always start by first doing something of value for each person/team… Build your posse.
It is good not to be alone.
If your boss is making you miserable, it is good to have a person (or five) of influence tell you that you matter.
The five outward looking reflections are more complex, and making changes recommended above requires far more influence and persuasion than you might imagine.
So many people I interview feel that all they are missing is authority to address issues we discussed today. I assure you, even with all the POWER and AUTHORITY in the world, for the issues we are discussing, you are going to fall short.
Your BFFs: Influence, persuasion and strategically placed incentives (which all of the above strategies are basically trying to do).
Bottom line.
There are leaders who want to know the truth, and welcome the truth in a blameless cultural spirit. I’m grateful when I’m in such environments.
When I don’t find myself in those environments, the recommendations above have helped me ensure A. I reduce the number of times Data/Analysts/I get scapegoated, and B. I ensure the collective focus remains on driving change that will improve customer love and business profitability.
Carpe diem!
The post Scapegoating Analysts | Recognizing & Preventing A Bad Idea. appeared first on Occam's Razor by Avinash Kaushik.
An extraordinary amount of time, effort, $$$ are spent on building dashboards/scorecards for CMOs… Yet, the end result, nearly always, is a useless data puke.
It turns out boiling the ocean is hard.
To build an effective big picture scorecard for the CMO, that is not data pukey, there are three crucial challenges that have to be solved:
Represent the full span of the CMO’s world. Marketing broadly tends to obsess about Paid Media, they have to care about Owned and Earned Media as well – the latter will be the source of 70% – 75% of the incremental conversions!
Deliver a roughly apples-to-apples comparison. How do you compare $10 spent on Paid Search vs. $10 spent on TikTok?
Build an evolutionary journey to nirvana. Every company is at a different point in their People, Process, Tools journey. CMO dashboards end up being useless when capabilities exceed needs.
In our strategic consulting practice at Croud, we work to solve all three of these problems, with the end goal being simple CMO dashboards that dramatically accelerate the journey to building a data-influenced organization.
In this article, I want to share the think that powers our initiatives in the hope that you’ll end up with fewer data pukes. In an approach that simplifies complexity, I’ll share the actual solution, with specific recommendations for KPIs and blessed methodologies, and do that inside a framework that allows for direct comparisons across dramatically different Marketing initiatives.
[Personal Bias: I prefer the word Scorecard over Dashboard. The latter is a fine word, but it has acquired such negative branding… I’ve switched to Scorecard, to get a fresh start. In my writing, in my keynotes, you’ll hear Scorecard.]
We’ll apply the methodology to Paid Media today – both Brand and Performance Marketing. If you are a TMAI Premium subscriber, or become one, you can also get the solution for additional Marketing initiatives – including Growth (Email, SEO, Referral, In-App Promos, Product Integrations), Events, Content Publishing, and Public Relations.
Setting the Foundation: Solving Comparability.
To build any big picture scorecard for a CxO, you need to solve the comparability problem. (#1 above.)
Perfection is impossible, but in our work we successfully switch from comparing apples to watermelons to hamsters to comparing green apples to red apples to yellow apples. Imperfect, but enough to allow us to make significantly smarter business decisions.
My solution is centered on organizing data/metrics/methodologies into a ladder of awesomeness (which solves for #3).
Here are the levels of sophistication in the ladder of awesomeness that’ll form the foundation of the CMO’s big picture scorecard:
Level No (Red): Most companies are here. Easy existence. Also, not all that great.
Level 1 (Yellow): At the minimum, focus on these metrics.
Level 2 (Green): These metrics/methods get you to learn actually useful things.
Level 3 (Blue): Analysis Ninjas live here. Hard stuff, massive and magical insights.
Level 4 (Black): Insanely cool and glorious business impact work that is certain to deliver a competitive advantage for your company.
This approach ensures that we do not lump together elements (metrics) that should never sit next to each other. Don’t mix Organic Social Impressions with Cost Per Individual Lifted (CPIL)!
The levels also ensure a clear-eyed assessment of where you might be today as an organization. They also help ensure that you appropriately distribute the work among the analytics resources you do have available. Agency does green, you do blue, no one does red. Or, as your resources/strengths dictate.
Finally, the levels will show that for some Owned, Earned, other initiatives… there are some questions you simply can’t answer. This is a good dose of reality – for example, for your Organic Search team that is “proving” to you the incremental impact of organic search investments via matched market tests!
Now that we have a simple ladder of awesomeness that solves for comparability, let’s go build ourselves marketing’s big picture dashboard.
Application #1: Paid Media CMO Scorecard Module.
Paid Media is typically broken into four clusters:
1. Brand-leading
2. Performance-leading
3. Channel Marketing (3rd parties to take your products to market)
4. Retail Marketing (Selling through a Target, Sainsbury's, Isetan)
For each, I’ll share recommendations for exactly what you should include in your CMO scorecard. The recommendations are in the format Metric | Methodology, what to measure and how to measure it.
For your Brand-leading initiatives, identify your level of awesomeness, and pick the Metric | Methodology for your CMO scorecard:
[Special Note: For simplicity’s sake, I’m skipping Level 4 – super advanced measurement. If you are a TMAI Premium subscriber, email me for edition #338 of the newsletter.]
Band-leading Marketing.
These are your big marketing initiatives on Television, TikTok, Hulu, The New Yorker. Typical objectives are to solve for metrics like Unaided Awareness, Consideration, Intent, (god forbid) Brand Love, etc.
Leverage true test-control methodologies as you measure Brand Lift (expressed as a percentage – Level 1), then # of People Lifted, and the delicious Cost Per Individual Lifted (Level 2). This ensures you can demonstrate incremental impact (yea!), and showing all three of those metrics add invaluable context to results delivered by your large brand dollars.
I recommend against using Pre-Post campaign surveys for any level, as they entirely lack the ability to causally identify the impact of brand-leading initiatives. I would like to say they are better than no measurement, but they really are not.
For Level 3, you’ll work to identify the impact of all the brand marketing you are doing at a portfolio level. So, not just measuring the lift from Facebook, lift from Hulu, lift from YouTube, lift from Billboards… But, the lift from all of them put together.
Why is Level 3 Analysis Ninja level? Because, you’ll usually use Matched Market Tests (MMTs) and they are fun but difficult. AND. You’ll discover that 4 points of lift from FB, 5 from Hulu, 6 from YT, and 7 from Billboards equals a portfolio level lift of 3 points!
You will never do brand marketing, and analysis, the same way again.
There is, of course, an entire ocean-full of brand metrics you can include. Leave them as diagnostic metrics for your agency, your creative team, and other contributing teams. The CMO simply needs the above to understand impact very close to the bottom line.
Performance-leading Marketing.
These are your PPC ads on Bing and Google, your AT&T TV commercials with a free iPhone 14 Pro, your catalogs, and BUY NOW ads on Instagram.
Lno and 1 are pretty straightforward.
Performance marketing has too much focus today on Overall Conversions and Cost Per Sale (or CPA). This is both irritating, and wrong, as it is claiming credit for results your Performance team/agency did not drive.
Hence, for Level 2, I recommend a focus on Incremental Conversions and Cost Per Incremental Sale (CPIS).
You can easily measure this at a channel level on all digital platforms using the built-in functionality of conversion lift studies (CLS). You can use matched market tests (MMTs) for non-digital platforms. [TMAI Premium subscribers, review how-to in editions #333, #334.]
If today you are reporting 10,000 Conversions from Search at a CPS of $85, be prepared to see 2,000 Incremental Conversions and a CPiS of $425! It is the reality check you desperately need to set higher expectations of your Performance team/agency.
To achieve Level 3 status, I recommend the same incrementality metrics, except at an x-stack level (to identify inefficiency by your Performance team/agency across platforms).
Ex: Today: 100 Incremental Conversions from Facebook, 120 Incremental Conversions from Instagram. When you measure x-Stack Incremental Conversions from Facebook AND Instagram put together, you’ll find the x-Stack Incremental Conversions = 90. (Or, 140 or 170.)
It is not hard to visualize just how dramatically your investment strategy will change, and with it will come far higher standards for your Performance team/agency to achieve.
[Note: If you are getting the feeling that Level 2 is the minimum viable point of existence, you are reading this article right. :) ]
Channel Marketing.
Channel marketing is third-parties taking your products and services, and bringing them to market. Say a company that makes construction equipment or IT products. There are private dealers and resellers who sell these products. Often, this reduces marketing costs for the manufacturer, while ensuring a greater diversity of marketing tactics and broader reach.
When we invest in channel marketing, we have less than optimal access to relevant data. Usually, all we see is money going out of the company, and all we can report is Spend by Channel Partner.
If you give this problem some thought upfront, you can write your contracts with incentives to get a minimum amount of data back. Ex: How are they spending your money, Spend by Channel.
You can use this to build trust across the ecosystem, and continue to ask for more data back that helps you identify Level 2 metrics, Attributable Sales (because your partners do have this data).
In my experience, there really is no option for Level 3 analysis. I welcome your suggestions.
In an exciting bit of development, if you spend material amounts on channel marketing, you can do Level 4 analysis for your Channel Marketing spend.
Retail Marketing.
You are Lysol.
A material chunk of your budget is being funneled into retail partners like Target, Costco, Walmart, and others. You are doing retail marketing! Those custom Apple stores inside Best Buy? Retail marketing.
You and I are likely not working for companies where we can get a lot of data back from our retail partners, in which case you might be stuck at Level No and report expenses like Spend on Fixtures.
Or, not that much of an improvement for Level 1 when we report data we do have Spend Per Partner. Though if you are lucky and a large volume is going through particular partners, you can always do welcome surveys – if your products allow for that engagement – and do some simple models to assess Sales by Partner.
As with Channel Marketing, you can build incentives for data exchange. At the very minimum, your channel partners will provide you with Return on Incentives and In-Store Sales, both really valuable for Level 2 analysis.
Level 3 is where there is a lot of fun, identifying incrementality. You will use MMTs to identify Incremental Sales and Cost Per Incremental Sale by, at least, each Retail partner.
Bonus: If your retail budgets are large, I recommend leveraging a/b/x experiments connected to your retail fixtures – content, layout, pricing, product mix – and use, in the US, DMA level slicing to understand the online and offline sales impact. Very cool.
Rest of the Story.
A CMO scorecard should empower them, as simply as possible, to understand the effectiveness and efficiency of the entire business of Marketing.
Hence, don’t stop with Paid Media, continue the journey… Here's your let me pick the very best KPIs for my CMO scorecard, depending on our level of Analytics sophistication chooser thingy…
[High-resolution image, right-click, "open image in new tab" or "save image as".]
And, here’s a special bonus… If your analytics practice is at Level 4 in the ladder of awesomeness, your CMO scorecard will look like this:
Simple. Powerful. Transformative.
It 1. completely covers all Marketing, 2. presents approx directly comparable KPIs, and 3. is at the level where you are in your analytics sophistication.
Not easy. But, nothing insanely profitable in life is easy. :)
[TMAI Premium subscribers, see #338, #339 on how to get to the above scorecard.]
Bottom line.
If your scorecard/dashboard is a data puke, the only winner is the person/agency who billed you tens/hundreds of thousands to build it.
In service of building a data-influenced CMO (/org), you will be strong, you will resist temptation, and you will build a scorecard that prompts strategic questions about Marketing’s incremental impact on profitability.
And, now you know how.
Carpe Diem!
The post Strategic Marketing Analytics: CMO Dashboards That Rock! appeared first on Occam's Razor by Avinash Kaushik.
Your analysis provides clear data that the campaign was a (glorious) failure. It could not be clearer. The KPI you chose for your brand campaign was Trust, it had a pre-set target of +5. The post-campaign analysis that compares performance across Test & Control cells shows that Trust did not move at all. (Suspiciously, there […]
The post Transform Data's Impact: Pick The Right Success KPI! appeared first on Occam's Razor by Avinash Kaushik.
If there is one thing the universe agrees on, it is that you should just provide data… You should provide INSIGHTS!!! In the 807,150 (!) words I’ve written on this blog thus far, at least 400,000 have been dedicated to helping you find insights. In posts about advanced segmentation, in posts about how to build […]
The post Winning With Data: Say No To Insights, Yes To Out-of-sights! appeared first on Occam's Razor by Avinash Kaushik.
Since you're reading a blog on advanced analytics, I'm going to assume that you have been exposed to the magical and amazing awesomeness of experimentation and testing. It truly is the bee's knees. You are likely aware that there are entire sub-cultures (and their attendant Substacks) dedicated to the most granular ideas around experimentation (usually […]
The post Robust Experimentation and Testing | Reasons for Failure! appeared first on Occam's Razor by Avinash Kaushik.
I was reading a paper by a respected industry body that started by flagging head fake KPIs. I love that moniker, head fake. Likes. Sentiment/Comments. Shares. Yada, yada, yada. This is great. We can all use head fake metrics to calling out useless activity metrics. [I would add other head fake KPIs to the list: […]
The post The Most Important Business KPIs. (Spoiler: Not Conversion Rate!) appeared first on Occam's Razor by Avinash Kaushik.
Almost all metrics you currently use have one common thread: They are almost all backward-looking. If you want to deepen the influence of data in your organization – and your personal influence – 30% of your analytics efforts should be centered around the use of forward-looking metrics. Predictive metrics! But first, let's take a small […]
The post Increase Analytics Influence: Leverage Predictive Metrics! appeared first on Occam's Razor by Avinash Kaushik.
One of the business side effects of the pandemic is that it has put a very sharp light on Marketing budgets. This is a very good thing under all circumstances, but particularly beneficial in times when most companies are not doing so well financially. There is a sharper focus on Revenue/Profit. From there, it is […]
The post Marketing Analytics: Attribution Is Not Incrementality appeared first on Occam's Razor by Avinash Kaushik.