We did a similar kind of experiment some time back and found both FB ads and Google ads were doing jack shit for us. Google was better. FB ads were absolutely useless. So we stopped our ad spend. We observed no change in our customer acquisition.
Another way these ad engines extract money out of you is by allowing competitors to bid for your brand name keyword. Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)
In our early days we used to be scared and bid for it.Lost a lot of money doing that. Then we talked about it and stopped. Again, no change in customer acquisition. Turns out, customers scroll down the ads and still click on our website.
Are our competitors getting some of our traffic?
Of course.
But my business is not built on the assumption that people will not find my competitors. Its built on the fact that people will explore options and try to find the best fit for them. So its ok.
If anything, FB ads were worse than useless for my business, all they attracted was 90% fraud.
Looking at what business Ozonetel is in (i.e. sales are directed towards business owners), I'm not suprised FB ads were also worthless for you. I get the sense FB ads are really only good for things a consumer might impulse buy (e.g. clothing, games, small gifts, etc.) but anything where the user is probably going to do some more in-depth thinking about the purchase, they're beyond useless. Which pretty much makes sense - even if you've drilled down your targeting well for business decision makers who may sign up for Ozonetel, it's highly unlikely that decision maker is going to really want to know more about Ozonetel while they're looking at cat vids and fighting with crazy uncles about politics.
Reading this thread I really start to feel people don't understand what fraud is. You shouldn't be calling it fraud when in the next paragraph you say that FB does work for impulse buys and not "ozonetel"
Just because your ads didn't convert, or 90% of the traffic you buy didn't convert, doesn't make it fraud. Fraud is malice, with the intention to defraud someone and not deliver what you promise, with intent. This is completely different to you setting a target demographic on FB and Google and it not converting.
I imagine most people calling fraud in this thread aren't actually talking about fraud, just non converting traffic. There could be a million reasons for this. You haven't set up conversion tracking, your targeting could be off, or your product or landing page is just crap. Calling it fraud is not helpful.
Facebook ads are the worst from a users perspective. I bought a floor mat through them once and for the next week all the ads were for floor mats. I just bought one I’m not interested in buying another. I bought a PC chair recently with my credit card, now I see a bunch of ads for PC chairs, again, I don’t need another one.
I work for a retail app and can usually tell when we have or haven't been spending on FB ads when I check our usage numbers for the week.
They do work but like anything it's about choosing the right tool (platform) for the job. We wouldn't run ads on LinkedIn and B2B SAAS companies shouldn't advertise on FB/Instagram.
Prefacing this with an acceptance both that HN culturally has US management principles and that I've got no idea how to run a business...
Why is it an initial assumption that customer acquisition and ad spend are linked short or even medium term? That doesn't seem right.
When I see an ad for washing machines it doesn't make me run out and buy a washing machine. It gets me familiar with Samsung the brand so in 6-12 months when I buy a washing machine I am familiar with Samsung and buy their brand. Or it means that if my washing machine starts to sputter I'm more confident knowing where to go to buy a new one.
It doesn't seem enough to stop advertising for 6 months and claim no damage is being done. It seems well understood that repeated exposure to a brand has a powerful effect on humans.
In online marketing/merchandising online advertising is very alluring for two reasons: immediate feedback and the few key players are well known. Essentially online advertising makes a marketing team’s life much easier even though it could be a net negative for the business revenue.
When I was doing some work in this space for Travelocity many years ago the biggest problem was cannibalism of data and traffic. Yes, the traffic coming in from ads was shitty and generally costs more than it returned, but even still cannibalism was the bigger problem. We would have been better off taking all the ads off the site and spending money in other offline venues. It was so apparent that ad spend was a drug addiction. I still don’t understand how this was a mystery to anybody in management considering the strength of the brand and the success of their television ads.
Eventually the business imploded and the partner/affiliate segment became more valuable than the core business partially because it was absent online ad spend.
You're correct on that. Large scale ad spend is divided into "branding" (creating awareness about the company and products) and "performance" (leading directly to a sale). The performance category has many factors including the path to the actual conversion/sale and how many steps are in the way and how long it takes. It can be instant (clicking a buy button in the ad itself) or many months.
Optimization is part of the process to ensure wasteful spending is reduced as analytics and sales data is generated but unfortunately the biggest issue is the people who actually run these campaigns. Lots of politics, perverse incentives, lack of skills, and general apathy.
Don't know how much of that is true as I am not a practitioner but when I was studying marketing we were taught this (continuous brand awareness reinforcement) only works for FMCG.
That's fine - that's awareness marketing and if you want to do it with online ads that's up to you. But then you should consider it compared to your other awareness marketing channel.
But the promise of performance marketing is exactly this immediate connection between ad spend and performance (i.e. sales).
Well then it becomes a question of attribution. How do you with a finite amount of dollars to spend on advertising know if you are getting your money's worth when you cannot connect a dollar spent on ad with incoming revenue?
Sure in broad strokes you are probably right that brand awareness has ROI positive in the long term. But then why would you buy targeted advertising vs a giant billboard on the main road?
Great initiative. Frankly most organisations continue spending on digital ads despite seeing diminishing returns because they have had hired people, often at the VP level to do just ads. These people obviously have to justify their salary so they forcibly convolute numbers to attribute some revenue to justify what they do- which is running ads.
They randomly change distribution models, ignore the bot traffic, and present eye candy line graphs to show "tremendous" impact from ads. And this keeps on running until cash graph goes below the threshold and CEO goes into the "introspection" mode.
One place where I worked spend a lot on ads, but they also hired some marketing people and a leads conversion person who seemed to be able to quantify how much return on investment they were getting from ads.
> Frankly most organisations continue spending on digital ads despite seeing diminishing returns because they have had hired people, often at the VP level to do just ads. These people obviously have to justify their salary so they forcibly convolute numbers to attribute some revenue to justify what they do- which is running ads.
How the hell do I structure my empire so that it out-lasts my reign as obsessive good emperor? I am the one guy who can finally say, “I’m sorry, you’re a good person and a great VP of marketing, but you’ve done such a great job that you’re now just marketing yourself really and you’re fired.”
> For ex, one ad network launched “battery saver” style apps in Google Play, giving them root access to your phone.
> When you type the word “Uber” into your Google Play, it auto-fires a click to make it look like you clicked on an Uber ad and attribute the install to themselves.
Isn't above an example of a criminal intent of deliberate fraud?
I think it's worth countering this with the fact that FB ads work very well for us at Thread, because in the HN echo chamber it's very easy to lose sight of this.
This is going to depend entirely on the product and the market. We're B2C, and a very "personal" service rather than a service for one's professional life, so that probably helps. Plus our target market isn't defined by being tech-savvy.
As with all channels, optimising for customer acquisition (not just clicks), and lifetime value (not just easy sales) is critical for the long term health of the channel.
We're probably not in your target market but most people here probably know anything marked with "ad" is very likely to contain malware and probably won't click it even if they think it goes where they want.
From my limited experience, resources are better spent to SEO to boost ranking. Also, you can request review on google ads keywords to prevent competitors from using your name, especially if you have a trademark.
Twitter has been by far the worst ad spend i've ever seen. When I complained that the clicks i got weren't real people, they blamed the landing page. I was advertising to 3 keywords "free astrophotography data" and the landing page was 0 ads, 0 popups and a link to free data to download.
They charged me for the ads, I complained to credit card company and did a charge back and they never followed through...
in fact, they offered me more credit to try advertising more.
For the life of me, I'm not sure how you can pay for 1,000 clicks to a very specific keyword and not have a single person download anything. The download was also valid for any user type. If you were on a mobile device, it showcased data that you could see on your mobile phone vs requiring a tool to download XISF or FITS data - it allowed you to download the high res image(s) in native iOS/Android formats.
Beyond the improbability of landing page being junk, Google analytics didn't fire for the majority of the traffic and twitters response was "the landing page may be too slow"
the landing page was < 1 second load times even within the twitter iframe nonsense.
anywhoo... the same ads on facebook, bing and google all had 75% click through and downloads but there were still a ton of clicks that had no analytics, no web time, didn't load anything in web analytics that we got charged for too...
That's at least something related being offered :)
When I searched for a very specific pillow type (and added country name to the search to get local results) I get this as first result on Google https://i.imgur.com/Fxx9b80.png. It's a mattress/pillow start-up/brand in India. The ad shows the exact thing I am looking for.
But when I click, and I am redirected to the actual product page https://i.imgur.com/eI6sNtJ.png that has none of these features and the product description is completely different from what it shows on Google Search result page.
I had reported this to Google sometimes back and nothing changed in their SEO gymnastics the last time I had checked a couple of weeks later.
I think what they are doing is fraud (wrong? unethical?). But what really annoyed me is it wasted a lot of my time as I thought I was doing something wrong and probably they actually had the product I am looking for.
Honestly, the way that ad companies treat brand specific search terms has a whiff of fraud to me. Surely Google is well aware that by the time an end user is searching for a brand, they have no interest in anything else; they just forgot the specific URL. If I ever found docs encouraging such behavior, I'd upgrade it from "whiff" to "pants on fire".
Too much to hope for: a Chromium fork to have the search hint dropdown provide suggestions + Duck Duck Go results. And maybe block the enter key unless a resolvable address had been entered to prevent inadvertent Google searches.
Serendipitous but of anecdata... was curious where you were located (ah, CA), so googled :) aaaand my keyboard decided I wanted, uhhh, "ozometel" (???).
Did you mean: ozonetel?
Nice.
(Gboard bit me again while typing it out here too lol)
> Another way these ad engines extract money out of you is by allowing competitors to bid for your brand name keyword. Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)
Fact checking you: On Google, I see at most a single ad (on most refreshes for Ozonetel, on one occasion for something called dialpad, on some cases, no ad is shown) and then your website, whatever it may be. It would seem that you are trying to piggyback on this article's discussion to increase awareness of your brand, which is not cool.
There was a freakonomics podcast recently about advertising (online and traditional).
No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
> No one is willing to run the experiments necessary
The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does.
The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don't want to know.
The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.
There are so many parties who want a slice of the brand's cash that they are all long past caring about whether the ad is being viewed by a human or not.
I work in the field. Incrementality testing is a big part of marketing measurement at any reputable agency. Any claims to the contrary are FUD.
That said, companies like P&G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.
Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.
Uber was just incredibly incompetent to not audit their ad spend at all.
I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn't easily externally visible, but tests like this are standard practice.
Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and/or incompetence.
> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that's bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they're awfully sure it does work (and perhaps even if they're right!).
Given that, I don't think the behavior has already given us an answer.
> The people that would have the power to run this experiment have their entire careers depending on things staying as-is.
Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results
It’s like saying that no one who works on reliability of the systems is willing to run experiment to throw 100% errors for a month, because running experiment like that may show that reliability of the systems doesn’t matter.
And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.
> The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Or they have run the experiment and the results haven't lines up with their own personal biases so they were discarded.
> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
Because advertising in some form certainly works. If you can determine that approach "A" that everybody is doing is actually a waste of money but approach "B" is effective, then you can develop services around approach "B" and market them based on these findings.
I don't understand this line of reasoning. P&G, Unilever, Cocacola, etc have never, not once in history, had a gung-ho C level exec who said "Screw it, I'm going to find out if our advertising works". And then either found it works and kept spending, or found out it doesn't work and saved literally billions of dollars.
There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi/Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi/frito. The idea that no one actually checked if this system works is simply not possible.
The trick isn't figuring out whether advertising in general influences people's behavior. The trick is in figuring out if any particular advertising campaign generated more profit than it cost to run.
Also, there's one alternative that's often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it's worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.
There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.
Fortune 500 is a cesspool of insane inefficiency balanced by equally ~~insane rent seeking~~ insanely secure revenue. The sooner everyone understands this, the better.
Part of the claim is that the people who are checking are ad execs who, if PepsiCo stopped buying ads, would shortly be out of a job (or have their budget and influence slashed). A counter to this might be that different advertising channels are likely not identically effective, and a TV ad exec has a big incentive to poke holes in non-TV ads.
How do you look into it though? I don't know, so i'm asking - but my immediate thought is that you treat it like science. You isolate an environment, advertise, and see if it has an affect. But the implication there is that if it doesn't have an affect, money is on the table.
If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They're throw as much money around as possible, and every cent lost or left on the table is panic inducing.
I also imagine different types or products and/or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don't know it exists.
For brands that large the majority of their spend isn't campaigns specific to new products, but overall brand management. The experiments would need to run for years or decades and if the prevailing belief that these are Red Queen's Races is correct, the cost if they're wrong could be the whole company.
> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.
This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).
This isn't to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn't performing well, and the company will adjust accordingly. It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
> without any attempt to validate their investments.
I've also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.
I have managed plenty of A/B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run "correctly", essentially nobody goes back and asks "wait, we had a 10% increase multiple times in the past year, but is our <metric> really showing the cumulative improvement we expected?"
The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.
I've worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.
I've talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.
Absolutely people are running "experiments" but the function of the "experiments" is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question "wait, does this really work? are these tests really able to capture the complexity of the environment?" And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.
Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.
> I work in adtech, and almost all companies run experiments in order to optimize their ad spend ... It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.
Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I'd doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.
While I'm sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don't even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.
That's great, thanks for your experience, but can't comment on the argument provided?
Isn't it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
>No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
This may be true but it's a separate issue than the fraudulent clicks.
Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn't get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It's an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.
However, this is one of those cases where despite being immeasurable, it still works.
Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can't do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It's not perfect.
And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.
Though not an intentional test, we found out what happens when a movie gets a wide release but does not advertise. In 2008, the movie 'Delgo' was released on over 2000 screens with nearly no advertising. Because the production company could not find a distributor, they spent their ad money on renting the screens for a week, with the hope that some people would randomly see it and word of mouth would spread, leading to the theaters wanting to keep it for additional weeks. It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening. More people saw Conan O'Brien making fun of the movie in his monologue than actually saw it in the theater.
The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there's no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.
> It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening.
On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?
I work in digital marketing and this episode made me want to tear my ears off.
First of all, they didn't differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.
And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it's really hard to get more than what you are paying for.
These experiments are run all the time even before the internet. I remember reading about how for broad brand advertising they used to segment by city. Half the cities got an ad for Coca Cola and half didn't. Then you compare sales of Coca Cola. No one will publicly publish numbers because it's a mix of sensitive sales data and competitive advantage (ie: otherwise your competitors don't have to burn money running their own tests). Also, smaller shops turn their online advertising campaigns on and off all the time to test impact.
> No one can actually prove it has any ROI at all.
That's a philosophical question of whether you consider statistics to be "proof".
> No one is willing to run the experiments necessary.
You're nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.
The Uber story is about venture capital and its anti-market incentives, not about the ad industry.
Read the podcast transcript. An economist proposed turning off print ads in one market to find out if they mattered, and the head of marketing said he’d rather not know.
> No one is willing to run the experiments necessary.
Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there's still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: http://www.davidreiley.com/papers/OnlineAdsOfflineSales.pdf
> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.
Depends enormously what sort of business you're in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A/B tests for every change.
This is nonsense. I had a startup completely powered by Google Ads. Ads brought in nearly 100% of the traffic. When my billing information with Google got mixed up and the ads stopped, the traffic went to 0 immediately.
This is incorrect. Companies do the experiments. They just don’t publish the results. Why would they? The idea that Amazon doesn’t know the ROI on their ad/marketing spend is laughable.
I'm only about halfway through the first of the podcast episodes you linked (thanks!), but just thinking about this logically for a moment:
I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I'm inclined to believe it has a huge effect on overall brand recognition.
Let's say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.
As I see it, buying brand advertising is a lot like buying an expensive suit. It's not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren't a fly-by-night company, and their products likely meet some standards of quality.
There's also the bit about luxury (mostly) car commercials. Half their purpose is to remind you that you made a good decision buying your <brand> car, and maybe your next car should be the same brand.
I'm a fan of the podcast but one argument they cited seemed to have a pretty glaring error - they looked at the case where eBay was comparing incremental gain on search ads over no ads. It's methodologically hairy because eBay is a very major player with significant brand recognition.
"When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay."
This doesn't show that advertising doesn't work per se, it shows that eBay didn't hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.
This was a superficial analysis at best. Adtech produces petabytes of data every day proving that advertising works. There's a reason why two of the most valuable internet companies sell ads.
The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it's only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can't then you were advertising to the wrong people in the first place.
Uber's mistake isn't that advertising didn't work, but rather that they didn't vet their vendors or even bother doing any checking and optimization of their own.
The implication here is that money spent on ads is largely wasted.
Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they're spending on ads - then it's very likely most others are in the same situation.
You are right that this doesn't mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.
We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It's too early to tell, but it's not impossible we'll see adspend drop significantly across the industry once everyone finds out they're just burning money.
This is patently false. I have friends who have worked years developing solutions for doing ad effectiveness comparisons. They are A/B tests in most cases but some methodologies require a lot of sophistication because of problems tracking conversions.
Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.
It's unclear how this experiment would be done. In the case of brand advertising, it's likely that brand awareness would decay over some period of time and in turn purchase behavior would change.
It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)
Come up with some new product that requires some personal data for usage (eg. age, gender/sex, address). Start to advertise this in just one country to one demographics, and look how many out-of-target orders you get.
Maybe it's even enough if you simply just sell it via mail order, you can then look at the addresses.
There's probably a natural information spread in any market (word of mouth, trade magazines), and there's probably a physical dispersion of the target group of people too (people move, visitors/tourists saw the ad/product and order it at home), but it still should be a valuable to see how much effect just one campaign has.
Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don't in others.
Traditional brand advertising testing (TV, newspaper, etc.) would be geography segmented as I understand it. So half the cities got the campaign and half didn't. You can mimic that with IP based geo-location although you'd get more leakage than pre-internet.
Counter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.
This is completely false. Large advertising platforms have many A/B tests that show significant differences in consumer behavior between groups that receive different ad treatments.
Ads might be less efficient than some believe, but it's super easy to see that they "work", and advertising platforms do it constantly.
"No one is willing to run the experiments necessary."
Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.
"Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold."
Telsa is in an enviable position of selling most of their cars before they're produced. In that position, you don't really need to advertise much. They also get a lot of PR to keep up brand awareness.
If Hyundai couldn't keep Genesis vehicles on the dealer lots, they'd advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.
Tesla's marketing spend is whatever it costs them in legal fees and fines to keep the mouthy celebrity CEO - and their high-profile campaigns in 2018 seemed pretty effective.
Beware extremes like "no one can actually prove". One difference between internet ads and their more ethereal tv and radio predecessors is that adviews, clicks and purchases can be tracked. Also, there are techniques that can make even TV, radio, print and even digital to physical world ad performance more visible: coupons, response codes and campaign-specific phone numbers and URLS. That Uber was buying hundreds of millions in ads and could not attribute performance (sales, for example) speaks more to poorly designed campaigns and potentially very bad actors in the supply chain.
Ads are somewhat like propaganda. There's no proving it will work or not in short terms, but nobody can deny that persistent propaganda has a long-term effect on the whole population. You have think about a kid/youth who listens to some propaganda for years when growing up. If say you show an ad for a new Apple product. A small part of it is to inform consumers about it, the longer larger part is to enforce the Apple brand which has been happening over many years. You can't say well it is not quite working so let's stop doing Ads.
Maybe not for companies of uber’s size/current reach, but small businesses definitely do benefit from ads. They see an immediate uptick in sales when they start advertising on various platforms.
Great subject matter, but boy does reading the transcription remind me why I hate podcasts. Stop infantilizing the audience with these coo-coo-ing sounds bites and get to the damn point already!
Lewis and Rao published a meta-analysis of 25 large scale controlled advertising experiment [1]. Here's the abstract:
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.
This story was on the Freakonomics podcast too. It definitely seems like a bubble, but there's no external pressure to make the bubble pop. With the mortgage crisis, eventually, people can't make the mortgage payments and the bubble bursts. But with people happily spending money on ads that don't work, there's no external pressure to stop. Will this bubble ever burst?
Gabriel Leydon of Machine Zone spoke to the general topic at Code/Media back in 2016. Basically discussing that they'd gone through the trouble of building internal expertise and tools for optimizing their ad spend to better ensure specific outcomes they desired/required in a way that would inevitably lead to more sophisticated ad buyers and putting a nail in the coffin of traditional media advertising.
This made the rounds in adtech in 2016 and it's really nothing special. His entire speech comes down to a single quote: "media will be quantified".
Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that's after dealing with all the politics and perverse incentives that happen in such a massive industry.
Alternatively, they dumped hundreds of millions of dollars into ads based on a wildly unrealistic notion of a customer lifetime (much like the rest of Silicon Valley).
Depending on what kind of advertising you are talking about. Direct response advertisers measure ads to the cents. And they know exactly what the ROI is and where the customer is coming from.
At least in e-commerce it’s very much possible. The company that I work for does such experiments regularly (there’s a dedicated Data Science team for measurement) and I’ve personally been involved in lift studies for Google Ads. They work, you just have to be careful with the ‘how much’ combined with ‘for what’.
Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).
Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!
Ecommerce sites have very fine grained measurement of their advertising spend and know exactly the ROI (which is why they focus so much on retargeting)
They mention the common retort to this which is very bizarre to me: "If online ads don't work, then huge companies wouldn't spend billions of dollars on them. Therefore they must work. You academics are just missing something."
Ok, how does those companies know it works? They don't have any real data to show that it does, just the fear that if they stop they'll lose a lot of business.
They do have data. It's pretty easy to see sales without ads vs sales with ads and then do further testing to narrow down results from there. This has been done for over a century since the first billboards were put up.
The amount of data generated by adtech today is staggering. The problem isn't data or advertising, it's the wrong people running the wrong campaigns for the wrong reasons.
I suspect that if our browser isn't blocking ads then our brain is. It's complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security/malvertising.
It's an arms race. In theory advertising should give you an edge over non-advertising competitors, but if everyone is doing it demand remains unchanged and also you're wasting money
The ad buyers aren't smart enough to measure the actual effectiveness of their ads, and the ad sellers are not incentivized to teach them how to do it. This can go on for an arbitrarily long time.
For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.
Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.
Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.
It'd be great to make experiments about how to sensible prevent/regulate this ad arms-race. But first better data privacy laws.
This is really, really not true. Advertising lift has been well studied, especially in metastudies spanning hundreds of digital campaigns across Facebook and Google. These are independent academic metastudies, with hundreds of millions of impression data samples.
Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.
Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.
Here are two of the foundational papers in this area:
Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.
You are conflating value of the marginal ad with value of any ads at all, and also exaggerating what those articles say
One of them said tracking cookies only boosted conversion 4%, and another said P&G did better with some traditional media advertising than some digital advertising.
This is now at 700 comments, and no one is talking about the math in the Twitter "thread". Found no mention in the Twitter replies either. Is it just me or is the math hilariously off?
[0]: "10% of $150M is $15M". Correct, but do we need a screenshot of a calculator for that, let alone Google? (why do people rely on an internet connection and Google for this?)
[1]: Talking about "2/3", but putting in "0.75". So which is it?
[2]: We are at $50M now. "Knock off" $20M, leaving us at $30M. Yet she puts in $20M into her Google calculator, gets 13.3% and calls it "a little over 1/10". Fair enough, but the real number is 30/150, so 1/5, 20%, double of "1/10".
Come on, this was not rocket science math! Should have double-checked before publishing, especially if you're a business consultant [3].
Yes, but marketers don't really care about math. Only about the numbers!
You would expect a few hiccups and a steep learning curve when the product they are trying to buy is a bid for visibility on insanely complex networks of platforms using said rocket science to guess where one's mind is at.
Disclaimer: worked years in the space on the topic of performance, only one time ever have i seen an AB test correctly run and correctly understood by a client. Most requested false numbers because they couldn't understand the difference between branding, attribution and incremental sales. To be fair, neither could most anyone working in the field... Seems like that doesn't change too quick!
As for the calculator screenshot, that kind of stuff feels like a consequence of having 'show your working' drilled into your head at school. Which is great to show your teacher that you understood the question and didn't plug it into a calculator, but unnecessary later on.
>[0]: "10% of $150M is $15M". Correct, but do we need a screenshot of a calculator for that, let alone Google? (why do people rely on an internet connection and Google for this?)
Edward Tufte seems to be here on earth to suffer for some offense in a previous lifetime. To know how badly statistics get abused, to be able to describe it, and be heard by a fraction of people who can't manage to fix the problem either.
No wonder he fucked off to a farm to concentrate on sculptures.
I use google for maths because it's the fastest thing I can get to and TYPE my maths in without having to open a new application. You can even just type it in the search bar of Chrome/Edge and see the result immediately
Both Windows and macOS support calculations in their respective system-wide search bars (Win key on Windows vs Cmd-Space for Spotlight on macOS). They both support pretty cool stuff like sqrt(9), 13^37, but Spotlight edges it slightly by also support bit shifting like 1 << 10 == 1024.
You think if every computer had bc in /bin, or the equivalent, most people wouldn't still be using Google for math?
It's not the lack of tooling, it's the lack of knowing how to quickly get to the tooling. For most people, especially non-technical, Win+"calc"+enter is not intuitive. Ctrl+T+$formula+enter is. That's the difference.
You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another.
You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.
All big companies do this.
Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on.
You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!!
Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either.
Edit:
Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).
They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.
This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics...
Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.
It's critically important for adtech companies' customers, since it's an essential part of determining return on investment.
But it's also critically important for both adtech companies and data scientists who work in the space to direct people's attention away from those sorts of metrics. You generally don't want to call the attention of the person who signs your paycheck toward the fact that it's all but impossible to really know for sure if your service has delivered them any net benefit.
The thing about the "advertising doesn't matter" argument is that there are situations where advertising absolutely matters - when a company is just starting out and there's no way they can get organic referrals, when the public is unaware of their existence, etc.
Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.
And if we look at those companies where maybe advertising actually doesn't help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they're ongoing businesses that probably reached that level through advertising and have money now to use for that.
But not only are they already advertising but they don't if their word-of-mouth/public-knowledge presence will last indefinitely, they don't know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn't offer any immediate increase in customers, continuing to spend on it doesn't seem to me as irrational as it sounds.
I think the implication here is "that honestly doesn't seem that important" because the expectation is that the number of people who would find you anyway would be a small number compared to a well-run ad campaign, at least over timeframes startups care about.
Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn't. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that's what the OP meant by it not seeming important: they're working under the assumption that "do any ads do anything" is clear and you don't need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don't perform well (which will also help save you from scammers).
One could argue that advertising for large brands with name recognition could be less valuable. I don't really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon's antics are, tbh, a form of marketing in and of themselves; "all press is good press."
Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you're a big brand vs a small one. But I'd be surprised if they're useless, at least in a competitive market. People don't have infinite money, so if they're looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn't, even if organically that sale might've gone to your company instead.
Also, I wouldn't argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary/useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.
Maybe the person is referencing how difficult it is to determine the value of the counterfactual? If it were possible to determine I imagine people would really want to know.
> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.
There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.
There’s an interesting question there about when this form of advertising becomes obsolete.
Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.
I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
Well, Uber is known to everybody by now. To the effect that "to Uber" is even kind of a verb/noun.
Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually.
It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.
And it's not like some new product, which without advertising nobody would even know it existed.
In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising.
So it doesn't sound strange that it could do without advertising today.
But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
There's a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.
Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.
>but you can also measure against SEO traffic with a high degree of certainty.
None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).
You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.
When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).
A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)
Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)
It would work for some of Uber's ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips.
So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations.
Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips.
In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.
Step 1) assume your ads won't work.
Step 2) have enough analytics / logging in place to convince yourself the ads do work.
Step 3) if they don't work, turn them off.
Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.
What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).
Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
You can definitely measure 'lift' - though it's far easier with online only sales.
For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out
One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.
For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...
Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features
Across this and the other[0] highly active digital ads post, there's 3 interesting forces at play:
a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry
b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)
c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out
(this is in jest, but I double-dare you to say it isn't at least directionally accurate)
Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way.
Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.
The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.
To finish with some constructive advice:
1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads.
2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.
3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.
It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative.
I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going.
If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.
"You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another."
Nope.
You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click/tap id. Android might. This is so inaccurate I don't know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this.
For native Android you 100% have access to an Android "device id". I set this up on an Android app we delivered. There's even companies that pay other Android apps to host their code inside the app, co-related that ID to other data available to the web (e.g. IP address, installed fonts, etc) to provide a very good guess of a user's device id via the web.
Apple had something similar, but you're right they're starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.
Back in the early 2010s I worked in ad tech on a data science team, and one of the things we were pushing for was causal A/B testing; basically turn off a campaign's advertising to a % of people and correlate it with sales to measure ROI.
As we were kicking this off I was at a conference chatting with an executive at another ad tech company. His response: "oh yeah I know a guy who tried that, he's not in the industry anymore."
We almost immediately came to realize our launch clients were getting negative ROI, sometimes severely so. AFAIK our efforts fizzled out, and I believe none of the people on my team are in the industry anymore.
Similar time frame for me, worked at an ad-tech startup, when retargeting was first becoming a big thing and we were pivoting the startup from the bad business idea it started to to doing retargeting as a demand"-side-platform" (DSP) on ad various exchanges. Tried myself to confirm whether any of it had any positive effect, I couldn't really discern any increase in the click-through-rate (CTR) for various approaches, but I'm not a stats expert, etc. so talked to the founder about how we should employ someone with a stats background. That convo went nowhere, and for that and other reasons I was out the door within a couple months.
That was the era of the ad-exchange DSP bubble. After that I went to work at another company that was on the other side of the exchange pipeline and I could see all these DSPs just plugging away doing their thing and none of it looked (to me) like it was accomplishing much. It was all bottomfeeding off of lower quality inventory but I suspect making big promises to investors.
That startup eventually pivoted a couple more times and sold to a bigger player a few years later, making some money for the founder but I suspect no value to the buyer.
Reading about startups that pivot dramatically multiple times make me cringe so hard.
The goal of pivoting isn’t throwing spaghetti at the wall until you hit something before running out of VC or angel money. That just shows terrible product/marketing leadership.
Your prior story is unsurprising combined with that.
No. A negative ROI just implies that the ratio of benefit/cost is less than one. If I incur a cost/investment of $100 but it creates value of $200 then I have an ROI of 100%. If this same expenditure instead only produced $80 value then I would have -20% ROI as in the value I’m realizing from my investment is 20% less than the cost of the investment.
As I read it, the amount they were spending on the ads themselves was more than the converted revenue from ad clicks. While the decline was probably (guessing) not linear, spending more on ads led to less than proportionally more revenue. If that had happened I can imagine calling that negative ROI.
Did they leave the industry because they realized their product wasn't providing a positive ROI, or were they fired for pointing that out to other people? I think the former is probably what you mean, I just want to be sure.
Yeah, it could be that many startups try to do advertising ethically, realize that it doesn't work, and drop out. That's necessarily going to leave all the firms willing to sell snake oil behind.
You can get firms selling stuff that doesn't work that are "trusted" simply because they've been around for years; there are plenty of distinguished brands selling homeopathic remedies, audiophile speaker cables, timeshares and MLM schemes.
Good question. In the first case it was left vague but I understood it as being fired and then not trying to find a new ad tech job. In the case of my team it was finding better industries to work in.
As a person with just shy of a stats graduate degree, the early 2010's were a nightmare of trying to get ad tech companies to grasp their incompetency. I met a dizzying number of slick talking frauds and became quite jaded.
Ok good, it wasn't just me being overly negative (because I usually am). I am not a stats grad, my math is weak, but it didn't look good to me, and I tried to get my employer to bring a stats specialist in to analyze. That got a cold response.
>Stopped [some] Spending on Digital Ads, Nothing Happened. Why?
[Added "some" for clarity.]
Let me try to restate the author's article because he presents it in a confusing way.
The issue is that both Google and Facebook have "core ad tech" that works decently with proven ROI -- and they both have "additional ad tech inventory" (a.k.a the partners/affiliates) that's much lower quality:
Facebook "quality" ad placements on their core platforms with decent ROI:
- ads in Facebook Newsfeed
- ads in Instagram feed
The "questionable" ad placements with much lower (possibly zero) ROI:
Same concept applies to Google AdWords. The adwords clicks on "google.com" perform better than the ones coming from partner websites. The lower quality ad tech in partner networks has more bots, more scam websites, more fraud, more negatives, etc that reduce ROI.
Bottom line is... if you're buying digital ads, you need to understand exactly what type of clicks you're paying for and how it actually performs.
It's not about clicks or traffic. The real issue is selection effect vs. advertising effect
SEO traffic benchmarks don't measure the right ting or show real ROI. FB or Google don't want you to measure the right thing.
Traffic to the website has licks, purchases and downloads that are happening anyway (selection effect) and clicks, purchases and downloads that would not have happened without ads (advertising effect) mixed. Only way to differentiate between the two is do real experiment where you divide target groups in two sections and show only the other group the ads.
When you do this experiment correctly, you find out that online ads are usually wasted money for any established brand and lower than expected value for less known brands.
These are pretty easy to measure on Google or FB with their "lift study" tools and a conversion pixel. The lift tools will let you create a holdout group of X% of people who should be targeted by your ad. They won't see the ad, but you can still measure the rate at which they convert on your site. Then, you can compare that conversion rate to the rate of people who actually saw/clicked the ad.
Conversion pixels are never 100% accurate (and are becoming increasingly inaccurate due to browser/OS changes) and shouldn't be used to say "This ad generated exactly X sales that wouldn't have happened otherwise!", but you can examine the relative rates from people who saw or didn't see your campaign.
I've also seen companies who don't trust FB/G to grade their own homework on a known lift study, so they measure the effect by running PSA ads. Their ad campaign will be something like "Donate to charity XYZ!" but will be set to measure purchases on their website. That lets them establish the baseline purchase rate of the customers in their audience.
That's more or less accepted. Ads that appear in Google search results have some value, since they appear when someone is looking for something specific. That's measurable.
Ads that appear on irrelevant web pages have less value. That's well known. Apparently so little value that it's near zero for known brands, this article says. In the end, they're like useless banner ads. Mostly clicked on by bots.
Just because the ad placement is low quality (eg audience network) doesn’t mean they can’t produce a positive ROI. Keep in mind, Lower quality comes with a lower price, so it makes sense for some brands, and some products. The reason people dont get a positive return on low quality ad placement is that they often use the same ad, or adverse the same product as for the higher quality ad placement. These ads and product often needs to be tailored for that specific placement.
There’s a useful correlary to this for consumer tech.
While it is tempting to monetize with a publisher network, these networks don’t work well enough to justify their own spend for many advertisers. As a publisher you’ll likely be stuck in a race to the bottom until you have sufficient scale for direct ad deals (read as 10s of million MAU).
I’m not sure if kicking fraud out of these platforms would raise ROI, or reveal that the real price of banner ads and interstitials is 0.
i usually get ads for things that i just bought. That means they are very good at snooping, on the other hand if I just bought a TV set then that doesn't mean that i would need another one soon.
Funny that Goole/Facebook currently don't seem to be running ads for complements of products too often; Amazon is better at that (if you bought this book, then you might also want this one).
I think a lot of this was already obvious to slightly-savvy mobile app users. Look at pretty much any free-with-ads app; a majority of the ads are going to be ads for other free-with-ads apps, many of which you already have installed. The thing is, outside of the Internet most ad spend is for marketing ubiquity, not direct response. You don't really buy, say, TV advertising with the expectation of getting so many clicks or calls out of it. And, up until a few years ago, TV was the lion's share of ad spend. So who knows if we'll see an actual industry reckoning or not. The whole point of advertising is to waste money, after all, and plenty of brands were fine with getting nothing but exposure out of it.
More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child/slave labor. It's a market for lemons; bad "money" (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?
The point of traditional advertising IS pretty much to waste money, similar to how the point of a peacock's tail feathers are to waste resources... it is a signal to potential partners (business or romantic) that they are so capable they can waste resources. It is a signal of bonafides.
I'm genuinely shocked at how many people think advertising doesn't work.
Just because it doesn't make a person consciously stand up and say, "I want a Snickers" does not mean the ad didn't work.
Like, honestly, are there people here suggesting viral ads don't work? Or don't understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.
MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn't advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).
Now, maybe that's what cortesoft is saying "get you to waste money" but if it got you to waste money on their product, it worked.
I'm not entirely sure about that, at least not in all cases.
I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.
Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as "literally a piece of foam, total crap mattresses." They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn't advertise at all, and was therefore cheaper.
So I guess in some markets, advertising does work. I've never heard of someone telling me how much they loved a mattress unless it's a Sleep Number or a Tempur Pedic, and those advertise by far the most.
This was true in the past, but nowdays it’s much easier to try out a product and see reviews.
With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.
Readit News
Note: this thread is being paginated, so you need to click More at the bottom of the page to read the rest of the comments. Or via these:
https://news.ycombinator.com/item?id=25623858&p=2
https://news.ycombinator.com/item?id=25623858&p=3
Another way these ad engines extract money out of you is by allowing competitors to bid for your brand name keyword. Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)
In our early days we used to be scared and bid for it.Lost a lot of money doing that. Then we talked about it and stopped. Again, no change in customer acquisition. Turns out, customers scroll down the ads and still click on our website.
Are our competitors getting some of our traffic?
Of course.
But my business is not built on the assumption that people will not find my competitors. Its built on the fact that people will explore options and try to find the best fit for them. So its ok.
Looking at what business Ozonetel is in (i.e. sales are directed towards business owners), I'm not suprised FB ads were also worthless for you. I get the sense FB ads are really only good for things a consumer might impulse buy (e.g. clothing, games, small gifts, etc.) but anything where the user is probably going to do some more in-depth thinking about the purchase, they're beyond useless. Which pretty much makes sense - even if you've drilled down your targeting well for business decision makers who may sign up for Ozonetel, it's highly unlikely that decision maker is going to really want to know more about Ozonetel while they're looking at cat vids and fighting with crazy uncles about politics.
Just because your ads didn't convert, or 90% of the traffic you buy didn't convert, doesn't make it fraud. Fraud is malice, with the intention to defraud someone and not deliver what you promise, with intent. This is completely different to you setting a target demographic on FB and Google and it not converting.
I imagine most people calling fraud in this thread aren't actually talking about fraud, just non converting traffic. There could be a million reasons for this. You haven't set up conversion tracking, your targeting could be off, or your product or landing page is just crap. Calling it fraud is not helpful.
They do work but like anything it's about choosing the right tool (platform) for the job. We wouldn't run ads on LinkedIn and B2B SAAS companies shouldn't advertise on FB/Instagram.
Why is it an initial assumption that customer acquisition and ad spend are linked short or even medium term? That doesn't seem right.
When I see an ad for washing machines it doesn't make me run out and buy a washing machine. It gets me familiar with Samsung the brand so in 6-12 months when I buy a washing machine I am familiar with Samsung and buy their brand. Or it means that if my washing machine starts to sputter I'm more confident knowing where to go to buy a new one.
It doesn't seem enough to stop advertising for 6 months and claim no damage is being done. It seems well understood that repeated exposure to a brand has a powerful effect on humans.
When I was doing some work in this space for Travelocity many years ago the biggest problem was cannibalism of data and traffic. Yes, the traffic coming in from ads was shitty and generally costs more than it returned, but even still cannibalism was the bigger problem. We would have been better off taking all the ads off the site and spending money in other offline venues. It was so apparent that ad spend was a drug addiction. I still don’t understand how this was a mystery to anybody in management considering the strength of the brand and the success of their television ads.
Eventually the business imploded and the partner/affiliate segment became more valuable than the core business partially because it was absent online ad spend.
Optimization is part of the process to ensure wasteful spending is reduced as analytics and sales data is generated but unfortunately the biggest issue is the people who actually run these campaigns. Lots of politics, perverse incentives, lack of skills, and general apathy.
But the promise of performance marketing is exactly this immediate connection between ad spend and performance (i.e. sales).
Sure in broad strokes you are probably right that brand awareness has ROI positive in the long term. But then why would you buy targeted advertising vs a giant billboard on the main road?
They randomly change distribution models, ignore the bot traffic, and present eye candy line graphs to show "tremendous" impact from ads. And this keeps on running until cash graph goes below the threshold and CEO goes into the "introspection" mode.
How the hell do I structure my empire so that it out-lasts my reign as obsessive good emperor? I am the one guy who can finally say, “I’m sorry, you’re a good person and a great VP of marketing, but you’ve done such a great job that you’re now just marketing yourself really and you’re fired.”
> When you type the word “Uber” into your Google Play, it auto-fires a click to make it look like you clicked on an Uber ad and attribute the install to themselves.
Isn't above an example of a criminal intent of deliberate fraud?
This is going to depend entirely on the product and the market. We're B2C, and a very "personal" service rather than a service for one's professional life, so that probably helps. Plus our target market isn't defined by being tech-savvy.
As with all channels, optimising for customer acquisition (not just clicks), and lifetime value (not just easy sales) is critical for the long term health of the channel.
Tried it. One ad. From ozontel.com
So Dialpad.
Pretty sneaky Dialpad lol
https://imgur.com/a/q70nShf
The problem was phishing attacks on our customers.
They charged me for the ads, I complained to credit card company and did a charge back and they never followed through...
in fact, they offered me more credit to try advertising more.
For the life of me, I'm not sure how you can pay for 1,000 clicks to a very specific keyword and not have a single person download anything. The download was also valid for any user type. If you were on a mobile device, it showcased data that you could see on your mobile phone vs requiring a tool to download XISF or FITS data - it allowed you to download the high res image(s) in native iOS/Android formats.
Beyond the improbability of landing page being junk, Google analytics didn't fire for the majority of the traffic and twitters response was "the landing page may be too slow"
the landing page was < 1 second load times even within the twitter iframe nonsense.
anywhoo... the same ads on facebook, bing and google all had 75% click through and downloads but there were still a ton of clicks that had no analytics, no web time, didn't load anything in web analytics that we got charged for too...
the whole thing seems like a scam
When I searched for a very specific pillow type (and added country name to the search to get local results) I get this as first result on Google https://i.imgur.com/Fxx9b80.png. It's a mattress/pillow start-up/brand in India. The ad shows the exact thing I am looking for.
But when I click, and I am redirected to the actual product page https://i.imgur.com/eI6sNtJ.png that has none of these features and the product description is completely different from what it shows on Google Search result page.
I had reported this to Google sometimes back and nothing changed in their SEO gymnastics the last time I had checked a couple of weeks later.
I think what they are doing is fraud (wrong? unethical?). But what really annoyed me is it wasted a lot of my time as I thought I was doing something wrong and probably they actually had the product I am looking for.
Did you mean: ozonetel?
Nice.
(Gboard bit me again while typing it out here too lol)
DHH (of RoR and Basecamp fame) has been beating the drum about this for a while now.
Fact checking you: On Google, I see at most a single ad (on most refreshes for Ozonetel, on one occasion for something called dialpad, on some cases, no ad is shown) and then your website, whatever it may be. It would seem that you are trying to piggyback on this article's discussion to increase awareness of your brand, which is not cool.
No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.
https://freakonomics.com/podcast/advertising-part-1/
https://freakonomics.com/podcast/advertising-part-2/
The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does.
The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?
A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don't want to know.
The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.
There are so many parties who want a slice of the brand's cash that they are all long past caring about whether the ad is being viewed by a human or not.
That said, companies like P&G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.
Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.
I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn't easily externally visible, but tests like this are standard practice.
Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and/or incompetence.
Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that's bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they're awfully sure it does work (and perhaps even if they're right!).
Given that, I don't think the behavior has already given us an answer.
Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results
And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.
Or they have run the experiment and the results haven't lines up with their own personal biases so they were discarded.
Deleted Comment
Because advertising in some form certainly works. If you can determine that approach "A" that everybody is doing is actually a waste of money but approach "B" is effective, then you can develop services around approach "B" and market them based on these findings.
It also happens to be the sole form of revenue all of the largest tech firms enjoy.
No wonder there hasn't been a real audit.
Also, all those marketing execs buying all the ads would be out of a job too, so they aren't speak up either.
There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi/Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi/frito. The idea that no one actually checked if this system works is simply not possible.
Also, there's one alternative that's often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it's worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.
There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.
If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They're throw as much money around as possible, and every cent lost or left on the table is panic inducing.
I also imagine different types or products and/or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don't know it exists.
Why?
Have definitely heard that one before. No disrespect.
Deleted Comment
This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).
This isn't to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn't performing well, and the company will adjust accordingly. It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.
I've also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.
I have managed plenty of A/B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run "correctly", essentially nobody goes back and asks "wait, we had a 10% increase multiple times in the past year, but is our <metric> really showing the cumulative improvement we expected?"
The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.
I've worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.
I've talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.
Absolutely people are running "experiments" but the function of the "experiments" is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question "wait, does this really work? are these tests really able to capture the complexity of the environment?" And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.
Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.
We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.
Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I'd doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.
While I'm sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don't even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.
Isn't it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.
This may be true but it's a separate issue than the fraudulent clicks.
Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn't get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It's an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.
However, this is one of those cases where despite being immeasurable, it still works.
Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can't do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It's not perfect.
And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.
The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there's no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.
On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?
Eminem released his last two albums without even announcing them. Word of mouth got them both to #1 on the charts.
If you tried that same movie experiment with something like Avengers, I bet the results would look a lot closer to Kamikaze than Delgo.
First of all, they didn't differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.
And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it's really hard to get more than what you are paying for.
Example: TVguide.com/listings. On most devices, I get a nondismissable audio ad when I'm looking at telecast schedules. I kill the audio immediately.
I never experience the bulk of the ad, and even if I can identify the product being advertised, I'll form a strongly negative impression of them.
I don't count this as "engagement", but it's charged as that.
That's a philosophical question of whether you consider statistics to be "proof".
> No one is willing to run the experiments necessary.
You're nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.
The Uber story is about venture capital and its anti-market incentives, not about the ad industry.
Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there's still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: http://www.davidreiley.com/papers/OnlineAdsOfflineSales.pdf
Depends enormously what sort of business you're in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A/B tests for every change.
I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I'm inclined to believe it has a huge effect on overall brand recognition.
Let's say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.
As I see it, buying brand advertising is a lot like buying an expensive suit. It's not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren't a fly-by-night company, and their products likely meet some standards of quality.
"When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay."
This doesn't show that advertising doesn't work per se, it shows that eBay didn't hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.
The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it's only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can't then you were advertising to the wrong people in the first place.
Uber's mistake isn't that advertising didn't work, but rather that they didn't vet their vendors or even bother doing any checking and optimization of their own.
Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they're spending on ads - then it's very likely most others are in the same situation.
You are right that this doesn't mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.
We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It's too early to tell, but it's not impossible we'll see adspend drop significantly across the industry once everyone finds out they're just burning money.
Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.
It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)
Maybe it's even enough if you simply just sell it via mail order, you can then look at the addresses.
There's probably a natural information spread in any market (word of mouth, trade magazines), and there's probably a physical dispersion of the target group of people too (people move, visitors/tourists saw the ad/product and order it at home), but it still should be a valuable to see how much effect just one campaign has.
Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don't in others.
Counter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.
Ads might be less efficient than some believe, but it's super easy to see that they "work", and advertising platforms do it constantly.
But none of them test "no ads" vs "ads".
Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.
https://www.motorbiscuit.com/gm-spends-an-embarrassing-amoun...
"Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold."
If Hyundai couldn't keep Genesis vehicles on the dealer lots, they'd advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.
[1] https://academic.oup.com/qje/article-abstract/130/4/1941/191...
https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...
Or, no one is willing to share the results of the experiments.
https://www.youtube.com/watch?v=oXBqzpExvrk
Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that's after dealing with all the politics and perverse incentives that happen in such a massive industry.
Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).
Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!
Ok, how does those companies know it works? They don't have any real data to show that it does, just the fear that if they stop they'll lose a lot of business.
The amount of data generated by adtech today is staggering. The problem isn't data or advertising, it's the wrong people running the wrong campaigns for the wrong reasons.
Because advertising is like military spending. It takes a lot of money to maintain the status quo.
I suspect that if our browser isn't blocking ads then our brain is. It's complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security/malvertising.
Deleted Comment
https://news.ycombinator.com/item?id=21465873
At worst, it’s a front for building “profiles” of everyone.
I can’t recall the last time I bought anything -because- of an ad.
If anything, ads have sometimes actually put me OFF from buying something!
For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.
Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.
Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.
It'd be great to make experiments about how to sensible prevent/regulate this ad arms-race. But first better data privacy laws.
https://news.ycombinator.com/item?id=25624112
“URL shorteners set ad tracking cookies”
This is incorrect.
Half was true in his time (a century ago) but I think our numbers are way worse.
I understand you no longer work there, but ads started in 2009 I believe, so you perhaps had some input on this?
https://www.japantimes.co.jp/news/2017/09/19/business/corpor...
Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.
Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.
Here are two of the foundational papers in this area:
- https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...
- https://www8.gsb.columbia.edu/media/sites/media/files/Garret...
Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.
They back up their claims with studies of their own as well as metanalysis.
One of them said tracking cookies only boosted conversion 4%, and another said P&G did better with some traditional media advertising than some digital advertising.
[0]: "10% of $150M is $15M". Correct, but do we need a screenshot of a calculator for that, let alone Google? (why do people rely on an internet connection and Google for this?)
[1]: Talking about "2/3", but putting in "0.75". So which is it?
[2]: We are at $50M now. "Knock off" $20M, leaving us at $30M. Yet she puts in $20M into her Google calculator, gets 13.3% and calls it "a little over 1/10". Fair enough, but the real number is 30/150, so 1/5, 20%, double of "1/10".
Come on, this was not rocket science math! Should have double-checked before publishing, especially if you're a business consultant [3].
0: https://twitter.com/nandoodles/status/1345786303263219712
1: https://twitter.com/nandoodles/status/1345792380184760321
2: https://twitter.com/nandoodles/status/1345794089825021963, https://twitter.com/nandoodles/status/1345795168105066496
3: https://twitter.com/nandoodles/status/1345853158279565313
You would expect a few hiccups and a steep learning curve when the product they are trying to buy is a bid for visibility on insanely complex networks of platforms using said rocket science to guess where one's mind is at.
Disclaimer: worked years in the space on the topic of performance, only one time ever have i seen an AB test correctly run and correctly understood by a client. Most requested false numbers because they couldn't understand the difference between branding, attribution and incremental sales. To be fair, neither could most anyone working in the field... Seems like that doesn't change too quick!
This is hilarious.
https://www.jacquescorbytuech.com/writing/marketers-addicted...
You can’t really blame academia for this one.
She knows her audience.
It's faster than opening a dedicated calculator program. If you're in a browser already, just ctrl-t and type your equation.
No wonder he fucked off to a farm to concentrate on sculptures.
What how much did they actually saved? Apology in advance because I really couldn't grasp what she was saying.
Deleted Comment
who cares!?
Because most computers lack a copy of bc in /bin.
It's not the lack of tooling, it's the lack of knowing how to quickly get to the tooling. For most people, especially non-technical, Win+"calc"+enter is not intuitive. Ctrl+T+$formula+enter is. That's the difference.
Dead Comment
You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.
All big companies do this.
Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.
But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on.
You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!!
Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either.
Edit:
Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).
They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.
This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!
Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.
But it's also critically important for both adtech companies and data scientists who work in the space to direct people's attention away from those sorts of metrics. You generally don't want to call the attention of the person who signs your paycheck toward the fact that it's all but impossible to really know for sure if your service has delivered them any net benefit.
Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.
And if we look at those companies where maybe advertising actually doesn't help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they're ongoing businesses that probably reached that level through advertising and have money now to use for that.
But not only are they already advertising but they don't if their word-of-mouth/public-knowledge presence will last indefinitely, they don't know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn't offer any immediate increase in customers, continuing to spend on it doesn't seem to me as irrational as it sounds.
Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn't. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that's what the OP meant by it not seeming important: they're working under the assumption that "do any ads do anything" is clear and you don't need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don't perform well (which will also help save you from scammers).
One could argue that advertising for large brands with name recognition could be less valuable. I don't really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon's antics are, tbh, a form of marketing in and of themselves; "all press is good press."
Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you're a big brand vs a small one. But I'd be surprised if they're useless, at least in a competitive market. People don't have infinite money, so if they're looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn't, even if organically that sale might've gone to your company instead.
Also, I wouldn't argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary/useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.
This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.
There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.
Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions.
I'd say there are about a million better uses of a million dollars than just pissing them away on a scam.
Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.
I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.
Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually.
It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.
And it's not like some new product, which without advertising nobody would even know it existed.
In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising.
So it doesn't sound strange that it could do without advertising today.
But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...
What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.
None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).
You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.
When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).
A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)
Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)
So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations.
Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips.
In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.
Step 1) assume your ads won't work.
Step 2) have enough analytics / logging in place to convince yourself the ads do work.
Step 3) if they don't work, turn them off.
Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.
What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).
Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.
For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out
One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.
For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...
Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features
a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry
b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)
c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out
(this is in jest, but I double-dare you to say it isn't at least directionally accurate)
Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way.
Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.
The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.
To finish with some constructive advice:
1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads.
2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.
3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.
0. https://news.ycombinator.com/item?id=25620707
Are they easy to categorize? Or is it a big secret what's dubious and what isn't?
There’s no revenue change even after testing (when you tack on costs of ad delivery).
A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative.
I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going.
If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.
Nope.
You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click/tap id. Android might. This is so inaccurate I don't know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this.
https://www.invoca.com/blog/what-is-idfa-and-why-apple-kille...
Apple had something similar, but you're right they're starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.
As we were kicking this off I was at a conference chatting with an executive at another ad tech company. His response: "oh yeah I know a guy who tried that, he's not in the industry anymore."
We almost immediately came to realize our launch clients were getting negative ROI, sometimes severely so. AFAIK our efforts fizzled out, and I believe none of the people on my team are in the industry anymore.
That was the era of the ad-exchange DSP bubble. After that I went to work at another company that was on the other side of the exchange pipeline and I could see all these DSPs just plugging away doing their thing and none of it looked (to me) like it was accomplishing much. It was all bottomfeeding off of lower quality inventory but I suspect making big promises to investors.
That startup eventually pivoted a couple more times and sold to a bigger player a few years later, making some money for the founder but I suspect no value to the buyer.
The goal of pivoting isn’t throwing spaghetti at the wall until you hit something before running out of VC or angel money. That just shows terrible product/marketing leadership.
Your prior story is unsurprising combined with that.
Feels like a missed something, as that sounds... counterintuivie.
(Not in any way related to the ad industry so pardon my ignorance)
You can get firms selling stuff that doesn't work that are "trusted" simply because they've been around for years; there are plenty of distinguished brands selling homeopathic remedies, audiophile speaker cables, timeshares and MLM schemes.
[Added "some" for clarity.]
Let me try to restate the author's article because he presents it in a confusing way.
The issue is that both Google and Facebook have "core ad tech" that works decently with proven ROI -- and they both have "additional ad tech inventory" (a.k.a the partners/affiliates) that's much lower quality:
Facebook "quality" ad placements on their core platforms with decent ROI:
- ads in Facebook Newsfeed
- ads in Instagram feed
The "questionable" ad placements with much lower (possibly zero) ROI:
- ads in Facebook Audience Network
(Here's an example screenshot in Facebook's Ads Manager to visualize the options above: https://storage.googleapis.com/website-production/uploads/20...)
Same concept applies to Google AdWords. The adwords clicks on "google.com" perform better than the ones coming from partner websites. The lower quality ad tech in partner networks has more bots, more scam websites, more fraud, more negatives, etc that reduce ROI.
Bottom line is... if you're buying digital ads, you need to understand exactly what type of clicks you're paying for and how it actually performs.
SEO traffic benchmarks don't measure the right ting or show real ROI. FB or Google don't want you to measure the right thing.
Traffic to the website has licks, purchases and downloads that are happening anyway (selection effect) and clicks, purchases and downloads that would not have happened without ads (advertising effect) mixed. Only way to differentiate between the two is do real experiment where you divide target groups in two sections and show only the other group the ads.
When you do this experiment correctly, you find out that online ads are usually wasted money for any established brand and lower than expected value for less known brands.
Conversion pixels are never 100% accurate (and are becoming increasingly inaccurate due to browser/OS changes) and shouldn't be used to say "This ad generated exactly X sales that wouldn't have happened otherwise!", but you can examine the relative rates from people who saw or didn't see your campaign.
I've also seen companies who don't trust FB/G to grade their own homework on a known lift study, so they measure the effect by running PSA ads. Their ad campaign will be something like "Donate to charity XYZ!" but will be set to measure purchases on their website. That lets them establish the baseline purchase rate of the customers in their audience.
Ads that appear on irrelevant web pages have less value. That's well known. Apparently so little value that it's near zero for known brands, this article says. In the end, they're like useless banner ads. Mostly clicked on by bots.
While it is tempting to monetize with a publisher network, these networks don’t work well enough to justify their own spend for many advertisers. As a publisher you’ll likely be stuck in a race to the bottom until you have sufficient scale for direct ad deals (read as 10s of million MAU).
I’m not sure if kicking fraud out of these platforms would raise ROI, or reveal that the real price of banner ads and interstitials is 0.
Funny that Goole/Facebook currently don't seem to be running ads for complements of products too often; Amazon is better at that (if you bought this book, then you might also want this one).
They are used for different reasons.
100% of ad buyers, marketing people know this, it's not an insight.
Nobody is 'accidentally spending money' on banner ads, when they thought they'd be spending money on Text ads.
Also - it's very straight forward to apply different measurements to FB ads in and out of network.
More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child/slave labor. It's a market for lemons; bad "money" (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?
Just because it doesn't make a person consciously stand up and say, "I want a Snickers" does not mean the ad didn't work.
Like, honestly, are there people here suggesting viral ads don't work? Or don't understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.
MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn't advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).
Now, maybe that's what cortesoft is saying "get you to waste money" but if it got you to waste money on their product, it worked.
I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.
Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as "literally a piece of foam, total crap mattresses." They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn't advertise at all, and was therefore cheaper.
So I guess in some markets, advertising does work. I've never heard of someone telling me how much they loved a mattress unless it's a Sleep Number or a Tempur Pedic, and those advertise by far the most.
With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.