I think its helpful to realize that the S&P 500 is not some static index of companies whose selection has been on auto-pilot but actually a fairly actively managed index whose selection algorithm is continuously being updated and refined. There is a not insignificant human judgement component of how to define the selection/weighting criteria.
In some senses, behind the scenes its likely similar to the Twitter/FB/Netflix algorithm - at some point there is some editorial opinion being inserted via criteria and weighing of those criteria.
"Some years ago Bill Miller, then a well-known Legg Mason portfolio manager with a stellar record of beating the market, noted that the S&P 500’s track record of being very hard to beat suggested that active management can succeed and that the Index Committee were actually good active managers."
Yes there's an index committee comprised of humans. But the index methodology is public and when there are tie break decisions needed, they're fairly predictable.
Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.
Is the algorithm "select * from companies order by market_cap desc limit 500"? At the risk of oversimplifying, there should be quite little to debate, and when some hair-splitting occurs, I trust a committee of reasonable humans will make a reasonable choice.
That's interesting. I was contacted by a personal money manager looking for clients and he explained his philosophy as sampling 30-50 stocks from the top end of the S&P 500. He had outperformed the market during a bull run, which I attributed to the increased risk of a smaller portfolio. I didn't invest because I figured his fund would underperform the index during the next bear run for that same reason, canceling out the gains and moreso with fees. Maybe there actually was something to his strategy; I haven't followed up on his performance.
The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value.
> * There is a not insignificant human judgement component of how to define the selection/weighting criteria.*
Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds.
It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US:
> The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more!
Agreed. I have ruined so many dinner conversations over this. You can't beat the SP500, it is the only free lunch. It is truly depressing when you think about it - you just feed the machine. The American millionare class is a group of 401k holders. [Oh. and they made 54.96% in the last 5 years]
They are not actually selling stock that's not performing well and buying stock that is, though, are they?
It's very hard to beat their performance if they can pick winners at any time and just discard losers without any actual loss.
This is going to change because too much money in indexes means not enough capital is chasing winners, and is instead enjoying the rising tide. That is okay and it is sustainable. But it still means that the upside for the ones who do chase winners and who succeed in placing correct bets on great companies will be more and more massive. Because the delta in capital from the index to the cream of the crop will be way larger in absolute terms as the indexes grow in size, and the winners will continue to prop up a huge rising tide of continued innovation.
I’d be worried if nobody could beat the market. 7% to 15% beating the market seems reasonable to me. The market is an incredible abstraction and it works, and it’s also still great and reassuring to know that there’s still an echelon of actively managed funds that beat the market.
Agree, ~7% seems ok? The funny thing about the profession of active fund management is that there is a ‘scoreboard’. Almost no other profession can or would score itself like this. If they did, I doubt more than 7% would beat a suitable benchmark for hairdressers, plumbers, doctors etc.
This ‘score’ has other useful properties. A successful fund manager has to operate under difficult and changeable conditions for long periods of time. They can’t get there just by default or by being lucky. Knowing which people can do this very challenging task means the advice they give about investing, managing your own psychology and life in general is imbued with a legitimacy that is difficult to find elsewhere.
>They can’t get there just by default or by being lucky.
Why do you say that? If I ran a League of Coinflip Guessers with 10,000 entrants, someone would still win the league, and probably with an astonishing score too. Why not the same with stock returns? You have 10k managers it stands to reason some of them will by pure chance beat whatever metric you set.
That's a very good insight. The success of the index funds is nothing but an indicator of the inflation from the top, that has not trickled down to Main Street until recently. In a stable solid monetary regime, capital is forced to pick individual stocks, as opposed to riding the inflation wave.
If you read the original bet and discussion carefully, what Buffett really bet against was the fees. It’s a very important detail in the original bet that the returns would be compared net of fees. His argument was not that active management was doomed to fail, but rather that active management is usually not worth what you pay hedge fund managers to do it.
Well he hasn't exactly bet against active management since all his wealth is actively managed. A more accurate interpretation would be "what I'm doing is incredibly difficult and I bet most who try will fail."
Given the sheer size of “mega cap stocks”—being Apple (AAPL), Microsoft (MSFT), Amazon (AMZN), Alphabet (GOOG & GOOGL), Meta Platforms (FB), Tesla (TSLA), NVIDIA (NVDA), and formerly Netflix (NFLX)—most portfolios carry noteworthy exposure to these companies.
When stripping out the entire cohort of Mega Cap Stocks, the S&P 500’s annualized performance in the five-year period ending February 28, 2022 would have fallen to 6.98% from 15.17% annually—a reduction of more than half. The index’s standard deviation would have seen an improvement of 4 percentage points, but the lost returns seem to sting more than the reduced volatility.
The pattern persists in each full calendar year since 2015. When each Mega Cap Stock is hypothetically excluded from the S&P 500’s total return, the index suffers. Notably, 2020’s 18.4% total return for the S&P 500 would have been a mere 3.3% had the eight Mega Cap Stocks been excluded, by our calculations.
Only in 2022 (year-to-date, through February 28, 2022) would removing the Mega Cap Stocks have actually boosted the S&P’s growth.
The point is that there's no way to know which "7" companies will carry the bulk of the growth, so you grab some representation of "all" of them.
So yeah, what you're saying is exactly how it's meant to go.
Also, "recession is looming" is easy to say at literally any time in history. Eventually, you'll be right. The trick is to know precisely when it will take place.
<5% of companies have driven most of the returns of equities:
> We study long-run shareholder outcomes for over 64,000 global common stocks during the January 1990 to December 2020 period. We document that the majority, 55.2% of U.S. stocks and 57.4% of non-U.S. stocks, underperform one-month U.S. Treasury bills in terms of compound returns over the full sample. Focusing on aggregate shareholder outcomes, we find that the top-performing 2.4% of firms account for all of the $US 75.7 trillion in net global stock market wealth creation from 1990 to December 2020. Outside the US, 1.41% of firms account for the $US 30.7 trillion in net wealth creation.
> Four out of every seven common stocks that have appeared in the CRSP database since 1926 have lifetime buy-and-hold returns less than one-month Treasuries. When stated in terms of lifetime dollar wealth creation, the best-performing four percent of listed companies explain the net gain for the entire U.S. stock market since 1926, as other stocks collectively matched Treasury bills. These results highlight the important role of positive skewness in the distribution of individual stock returns, attributable both to skewness in monthly returns and to the effects of compounding. The results help to explain why poorly-diversified active strategies most often underperform market averages.
The trick is knowning which company(ies) will be that 5% and when: some do well for a time and then fade away: you would have to know when to jump in and out of them.
Also, the S&P 500 being concentrated is not new and has been the case for 40+ years:
If you had invested in a fund that just tracked S&P 500, what would have been your, over, say, the last ten years, what would have been your returns? I can't figure out if this info is in the article.
Are there low-fee funds that just track the S&P 500? That would seem to be the way to go for long-term investing? What such funds can I find? (taking into account as other comments in this thread point out that, yes, the S&P is actually actively managed itself, sort of).
I'm continually astounded by techies who have so much money and so little idea what to do with it. It's not like this stuff is even hard, not compared to keeping up the latest JS nonsense: https://www.bogleheads.org/wiki/Main_Page#mp-gs-h2
You may be over-estimating how much money I have as a techie, I work in non-profit/academic sector. My retirement funds are relatively paltry compared to what you may assume about techies with so much money (but perhaps still more than US median for my age; googling says median US retirement savings across all working-age households is $95,776), and mostly in managed 401k/403b where I have previously just chosen "lifecycle funds". ¯\_(ツ)_/¯
People make a lot of assumptions about how much wealth random HN commenters or the HN audience in general have, and sometimes I start feeling like I'm really poor compared to all you techie multi-millionaires... but when I actually pay attention to comments, I realize, nope, a lot of HN is more like me (although probably often ashamed to say so).
"...If you had invested $10,000 in the S&P 500 index in 1992 and held on with dividends reinvested, you'd now have more than $170,000. The market volatility in 2022 could cause this return to decline somewhat. However, the index has proven to be a winner over the long term..."
OK thanks! now to figure out how to calculate that as an annual rate of return (APY?) Sounds like this was through 2021, already a bit old?
If I plug this into a random calculator on the web without knowing what I'm doing, $10K to $170K over 20 years looks like... around 15.25%? That does seem quite high, can that be right?
In a basic theoretical model excluding things like individual holders, short term speculators and stocks outside of the index, if you add up all the active funds, then the holdings should indeed match the passive funds (since the passive funds are just holding a percentage of the total, the remainder must hold the remaining percentage).
So in this toy scenario obviously active funds would underperform after fees. But there's more reason for investors to pick stocks than just beat the market (risk preferences - for a lot of people the benchmark is not the S&P but bond yields). The majority of active management is not hedge funds and are not in the business of providing outsized returns (especially not to small retail clients). To beat the market, one must take risks that the other active managers or index committees deem to be unacceptable.
Readit News
In some senses, behind the scenes its likely similar to the Twitter/FB/Netflix algorithm - at some point there is some editorial opinion being inserted via criteria and weighing of those criteria.
"Some years ago Bill Miller, then a well-known Legg Mason portfolio manager with a stellar record of beating the market, noted that the S&P 500’s track record of being very hard to beat suggested that active management can succeed and that the Index Committee were actually good active managers."
https://www.indexologyblog.com/2014/08/07/inside-the-sp-500-...
Another thing to think about is that most of the outperformance of the index is driven by the largest companies. But not because they were added when they were their current size. But because of the buy and hold approach of getting them in when they're still on the small end of the large caps. There was a good recent paper on this concept, showcasing that most of the big index returns are attributable to only a few stocks. Similar to how most of global market returns are mostly just US.
Here's the current breakdown:
Apple 7.08%, Microsoft 6.20%, Amazon.com 2.62%, NVIDIA 1.87%, Alphabet 1.84% Berkshire 1.65% Alphabet 1.61% Tesla 1.44%
which if it was an active manager would be highly opinionated.
The SP500 index might be a little more complicated than just the 500 biggest public companies. When Tesla was rejected despite meeting criteria, it was odd. However “edition” impacts less than 1% the actual value.
If you can compare VOO (Vanguard sp500 tracking) vs VV (Vanguard 566 large cap), it’s the same: https://www.etf.com/etfanalytics/etf-comparison-tool/VOO-vs-... (20 basis-point difference)
Not wrong, but the rules are relatively fixed and known ahead of time and somewhat more deterministic compared to the decision-making process of most active funds.
It should also be noted that the S&P 500 isn't the only index. The Russell 3000 and Wilshire 5000 try to cover "all" publicly trade companies in the US:
* https://en.wikipedia.org/wiki/Russell_3000_Index
* https://en.wikipedia.org/wiki/Wilshire_5000
But "passive" investing does exist on a spectrum:
> The terms passive investing and index investing are often intertwined, but they are not exactly the same thing. Today’s guest is Adriana Robertson, the Honourable Justice Frank Iacobucci Chair in Capital Markets Regulation […].Adriana is interested in index investing and, in this episode, we hear her views on whether or not index investing is passive. Hear facts from her paper on the S&P 500 Index fund specifically, and all of the reasons that it's not passive, as well as some of the issues that are potentially arising from the creation of so many indexes or so-called passive investments. A more recent paper by Adriana, published in The Journal of Finance, surveyed a representative sample of U.S. individual investors about how well leading academic theories describe their financial beliefs and decisions, and Adriana shares the differences in something like value growth from an academic perspective versus a real-world perspective. Find out how investors can go about evaluating the performance of their portfolios and what they should be looking for when deciding which index fund to invest in, as well as why index funds aren’t a meaningful category anyway, factors from Adriana’s surveys that might influence investor’s equity allocation, and the trend towards indexing and whether it will overtake active portfolios. Tune in today for all this and more!
* https://rationalreminder.ca/podcast/133
What do you mean by this? That pension saving make up the bulk of wealth in the US? Because from what I can gather that's not correct.
Deleted Comment
It's as if researchers complained that placebo is too therapeutic.
I’d be worried if nobody could beat the market. 7% to 15% beating the market seems reasonable to me. The market is an incredible abstraction and it works, and it’s also still great and reassuring to know that there’s still an echelon of actively managed funds that beat the market.
This ‘score’ has other useful properties. A successful fund manager has to operate under difficult and changeable conditions for long periods of time. They can’t get there just by default or by being lucky. Knowing which people can do this very challenging task means the advice they give about investing, managing your own psychology and life in general is imbued with a legitimacy that is difficult to find elsewhere.
Why do you say that? If I ran a League of Coinflip Guessers with 10,000 entrants, someone would still win the league, and probably with an astonishing score too. Why not the same with stock returns? You have 10k managers it stands to reason some of them will by pure chance beat whatever metric you set.
https://longbets.org/362/
https://www.forbes.com/sites/dereksaul/2023/04/10/these-7-te...
Everything else is still down and recession likely glooming. And when the recession hits the massive gainers are likely getting same-size correction.
The article was written in mid-2022. from https://get.ycharts.com/resources/blog/whats-the-sp-500-with...:
Given the sheer size of “mega cap stocks”—being Apple (AAPL), Microsoft (MSFT), Amazon (AMZN), Alphabet (GOOG & GOOGL), Meta Platforms (FB), Tesla (TSLA), NVIDIA (NVDA), and formerly Netflix (NFLX)—most portfolios carry noteworthy exposure to these companies.
When stripping out the entire cohort of Mega Cap Stocks, the S&P 500’s annualized performance in the five-year period ending February 28, 2022 would have fallen to 6.98% from 15.17% annually—a reduction of more than half. The index’s standard deviation would have seen an improvement of 4 percentage points, but the lost returns seem to sting more than the reduced volatility.
The pattern persists in each full calendar year since 2015. When each Mega Cap Stock is hypothetically excluded from the S&P 500’s total return, the index suffers. Notably, 2020’s 18.4% total return for the S&P 500 would have been a mere 3.3% had the eight Mega Cap Stocks been excluded, by our calculations.
Only in 2022 (year-to-date, through February 28, 2022) would removing the Mega Cap Stocks have actually boosted the S&P’s growth.
So yeah, what you're saying is exactly how it's meant to go.
Also, "recession is looming" is easy to say at literally any time in history. Eventually, you'll be right. The trick is to know precisely when it will take place.
- They are making $40/year avg. off a user by selling users data
- They announced massive stock buybacks
Not sure if this is the best capital allocation for society - would love to see less parasitic platforms to success.
Not really sure why people complain so much about stock buybacks. They're tax advantaged dividends.
People need to stop parroting this. Meta targets ads with user data. That's very different than selling user data.
Just to be clear, we're in a recession and have been for a while now, despite government administrators attempts to change the definition.
By what possible criteria are you using to assert we’re currently in a recession?
Exxon Mobil (XON) was the #3 company in 2001, but not even in the top twenty in 2021:
* https://www.bespokepremium.com/think-big-blog/largest-25-sto...
<5% of companies have driven most of the returns of equities:
> We study long-run shareholder outcomes for over 64,000 global common stocks during the January 1990 to December 2020 period. We document that the majority, 55.2% of U.S. stocks and 57.4% of non-U.S. stocks, underperform one-month U.S. Treasury bills in terms of compound returns over the full sample. Focusing on aggregate shareholder outcomes, we find that the top-performing 2.4% of firms account for all of the $US 75.7 trillion in net global stock market wealth creation from 1990 to December 2020. Outside the US, 1.41% of firms account for the $US 30.7 trillion in net wealth creation.
* https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3710251
> Four out of every seven common stocks that have appeared in the CRSP database since 1926 have lifetime buy-and-hold returns less than one-month Treasuries. When stated in terms of lifetime dollar wealth creation, the best-performing four percent of listed companies explain the net gain for the entire U.S. stock market since 1926, as other stocks collectively matched Treasury bills. These results highlight the important role of positive skewness in the distribution of individual stock returns, attributable both to skewness in monthly returns and to the effects of compounding. The results help to explain why poorly-diversified active strategies most often underperform market averages.
* https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2900447
The trick is knowning which company(ies) will be that 5% and when: some do well for a time and then fade away: you would have to know when to jump in and out of them.
Also, the S&P 500 being concentrated is not new and has been the case for 40+ years:
* https://awealthofcommonsense.com/2020/02/5-companies-make-up...
It's just the companies have changed (and there's no way of easily of predicting which ones did / will rise and fall).
Are there low-fee funds that just track the S&P 500? That would seem to be the way to go for long-term investing? What such funds can I find? (taking into account as other comments in this thread point out that, yes, the S&P is actually actively managed itself, sort of).
...yes. Sorry, I don't mean to sound like a jerk, but is this really a question? There are dozens of them: https://www.thebalancemoney.com/the-cheapest-sandp-500-index...
I'm continually astounded by techies who have so much money and so little idea what to do with it. It's not like this stuff is even hard, not compared to keeping up the latest JS nonsense: https://www.bogleheads.org/wiki/Main_Page#mp-gs-h2
You may be over-estimating how much money I have as a techie, I work in non-profit/academic sector. My retirement funds are relatively paltry compared to what you may assume about techies with so much money (but perhaps still more than US median for my age; googling says median US retirement savings across all working-age households is $95,776), and mostly in managed 401k/403b where I have previously just chosen "lifecycle funds". ¯\_(ツ)_/¯
People make a lot of assumptions about how much wealth random HN commenters or the HN audience in general have, and sometimes I start feeling like I'm really poor compared to all you techie multi-millionaires... but when I actually pay attention to comments, I realize, nope, a lot of HN is more like me (although probably often ashamed to say so).
"...If you had invested $10,000 in the S&P 500 index in 1992 and held on with dividends reinvested, you'd now have more than $170,000. The market volatility in 2022 could cause this return to decline somewhat. However, the index has proven to be a winner over the long term..."
I.e. the entry point matters a lot.
If I plug this into a random calculator on the web without knowing what I'm doing, $10K to $170K over 20 years looks like... around 15.25%? That does seem quite high, can that be right?
So in this toy scenario obviously active funds would underperform after fees. But there's more reason for investors to pick stocks than just beat the market (risk preferences - for a lot of people the benchmark is not the S&P but bond yields). The majority of active management is not hedge funds and are not in the business of providing outsized returns (especially not to small retail clients). To beat the market, one must take risks that the other active managers or index committees deem to be unacceptable.