Warren Buffett and Berkshire Hathaway have one stock that's clearly their favorite (and it's not even close). Apple (AAPL -0.35%) makes up nearly 48% of Berkshire Hathaway's portfolio, with a stake worth nearly $180 billion.

That's an extreme concentration, and Buffett is OK with that. He's often stated that diversification doesn't make sense if you know what you're doing. But with the direction that Apple is heading, this concentration may bite Berkshire in 2024 -- and Apple may be a stock for potential investors to avoid now.

Apple was a deal in 2016, but it's expensive now

Berkshire's first purchase of Apple stock was in the first quarter of 2016. At that time, the iPhone 6 was the latest model, which was already quite popular among consumers.

Apple was also hit by a sales downturn in 2016, which didn't give investors much confidence in the stock. As a result, Buffett and Berkshire decided to purchase the stock. The combination of strong brand value, a technological shift, and a cheap stock was too much to ignore.

In Q1 of 2016, Apple's stock had an average valuation of about 10.6 times earnings. Looking back, it seems amazing that Apple could trade that cheaply, but it was right under many investors' noses.

As Apple grew, Berkshire added more Apple stock, which was a great decision. While Apple's business certainly grew, the stock's valuation also rose through a mechanism known as multiple expansion, which is when investors are willing to pay more for a business than they previously were. Because of this, Apple's stock has grown to become incredibly expensive.

AAPL PE Ratio Chart

AAPL PE Ratio data by YCharts

Apple's growth doesn't match its valuation

At the same time, its business isn't excelling these days. The rise of Apple has been tied to the iPhone. Each year, Apple seemed to introduce new, innovative technology that caused millions to upgrade. However, the performance jump from generation to generation has slowed, making upgrading less enticing.

Furthermore, inflation has reduced consumer spending power. This has caused the sales of iPhones (as well as other Apple products) to slip or become fairly stagnant.

Quarter iPhone Sales YOY Growth
Q1 FY 2023 (8.1%)
Q2 FY 2023 1.4%
Q3 FY 2023 (2.5%)
Q4 FY 2023 2.1%

Data source: Apple. FY 2023 ended Sept. 30, 2023. YOY = year over year.

All eyes will be on Apple when it reports its fiscal 2024 first-quarter results, including the all-important holiday quarter. These results will likely be available in early February and could set the trend for Apple's stock in 2024.

Another headwind for Apple is its Apple Watch. Due to patent infringement, Apple had to pull its Watch Series 9 and Ultra 2 from stores and online before Christmas Day. While it's unknown how the dispute with Masimo (the patent owner of light-based pulse oximetry sensors) will shake out, this could greatly affect Apple's finances if it has thousands of watches with technology it cannot sell.

Wall Street analysts aren't high on Apple stock either. They expect revenue growth of 3.5% and 5.6% in fiscal 2024 and 2025, respectively. But revenue growth isn't everything. The same analysts also project 7.2% and 8.8% earnings per share growth, which would still be market-underperforming growth.

Even though Apple is slated to grow less than 10% for the next few years (per analysts), the stock trades like a premiere growth stock. These two aren't compatible and could cause issues with Apple stock for the next few years. As a result, I'm staying far away from Apple stock. Berkshire doesn't have that luxury as it would incite a massive panic if it began dumping shares.

Concentration is great when it works, but it can also work against you if you're as large as Berkshire and you can't easily exit a position.