Put simply, Warren Buffett has created one of the most remarkable companies ever through Berkshire Hathaway (NYSE:BRK-A)(NYSE:BRK-B).

But there is one major threat that could bring the castle he created crumbling down.

One cause of concern
There's a lot to like about Berkshire Hathaway and the empire Warren Buffett has built, but its biggest threat may in fact be just that, it's empire. Or, more exactly, what the Federal Reserve decides to do with that empire.

Earlier this year, we learned regulators had begun to look at Berkshire Hathaway -- and its nearly half a trillion dollars in assets -- to determine whether or not it should be deemed a "Systematically Important Financial Institution," or SIFI. If the Financial Stability Oversight Council determines Berkshire Hathaway is a SIFI -- meaning its failure could threaten the broader stability of the markets -- it could be a devastating blow to Buffett's company.

This designation would result in new regulatory scrutiny and oversight. It would become regulated by the Federal Reserve and as the consultancy Deloitte notes, "Nonbank SIFIs must comply with the Board of Governors' capital plan rule and develop annual capital plans, conduct capital adequacy stress tests, and maintain adequate capital."

In short, it would face the same continuous oversight and requirements many have worried may cripple the ability of banks to earn adequate returns. It was just in December of last year when Bloomberg reported insurer MetLife revealed "pending oversight from U.S. regulators poses the biggest challenge," to its goal to increase its profitability.

The major impact
While we can only speculate on what the impact could be to Buffett and Berkshire Hathaway, it's not unreasonable to think this designation would lead to increased costs as Berkshire maintained its newfound requirements. With just 24 corporate employees, one has to think more would need to be hired to simply comply with the reporting.

Not only would it lead to greater expenses, it could also potentially limit its ability to make massive investments as deploy its earnings to the most profitable businesses. And that is to say nothing of the limit that could be imposed on its ability to make acquisitions of other businesses, which stood at a resounding $21 billion in 2013 alone.

In the 2009 letter to Berkshire Hathaway shareholders Buffett said:

We will never become dependent on the kindness of strangers. Too-big-to-fail is not a fallback position at Berkshire.

We can only hope the government agrees with him.